2024-05-08T03:56:46.007 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-08T03:56:46.007 DEBUG:teuthology.run:Teuthology command: teuthology --name kchai-2024-05-08_00:43:58-rados-wip-kefu-testing-2024-05-06-distro-default-smithi --description rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} tasks/libcephsqlite} --owner scheduled_kchai@teuthology --verbose --archive /home/teuthworker/archive/kchai-2024-05-08_00:43:58-rados-wip-kefu-testing-2024-05-06-distro-default-smithi/7696821 -- /home/teuthworker/archive/kchai-2024-05-08_00:43:58-rados-wip-kefu-testing-2024-05-06-distro-default-smithi/7696821/orig.config.yaml 2024-05-08T03:56:46.043 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T03:56:46.131 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/kchai-2024-05-08_00:43:58-rados-wip-kefu-testing-2024-05-06-distro-default-smithi/7696821 branch: wip-kefu-testing-2024-05-06 description: rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} tasks/libcephsqlite} email: null first_in_suite: false job_id: '7696821' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: kchai-2024-05-08_00:43:58-rados-wip-kefu-testing-2024-05-06-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-kefu-testing-2024-05-06 ceph: conf: client: debug cephsqlite: 20 debug client: 20 debug ms: 1 global: mon client directed command retry: 5 mon election default strategy: 3 ms bind msgr1: false ms bind msgr2: true ms inject socket failures: 5000 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub interval: 300 paxos service trim min: 10 osd: bluestore block size: 96636764160 bluestore compression algorithm: zstd bluestore compression mode: aggressive bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - POOL_APP_NOT_ENABLED - do not have an application enabled sha1: 5ccf1c30cf724d5371f55439b3b3a90cb994fae2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 5ccf1c30cf724d5371f55439b3b3a90cb994fae2 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-kefu-testing-2024-05-06 sha1: 5ccf1c30cf724d5371f55439b3b3a90cb994fae2 owner: scheduled_kchai@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 2477 sha1: 5ccf1c30cf724d5371f55439b3b3a90cb994fae2 sleep_before_teardown: 0 subset: 1030/99999 suite: rados suite_branch: wip-kefu-testing-2024-05-06 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_5ccf1c30cf724d5371f55439b3b3a90cb994fae2/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 5ccf1c30cf724d5371f55439b3b3a90cb994fae2 targets: smithi170.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDISdccAMpjTTpR17SreXiq5NUAYaHMBL61dw0VrbGVPmisU3BYRNBac0cDFl5tw2vHmlDZ77nLl7Dvsv2O1tW8= smithi187.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA9FW+CA8Pfrhp6UvcPN2OnV1xZvW3bofLOIQf+CZelvy60JgzQQ8NVquLstj4GVp5uSLomwvG5Qy1KSQFa3+ig= tasks: - install: extra_system_packages: deb: - sqlite3 rpm: - sqlite-devel - ceph: null - exec: client.0: - ceph osd pool create cephsqlite - ceph auth get-or-create client.libcephsqlite mon 'profile simple-rados-client-with-blocklist' osd 'allow rwx pool=cephsqlite' >> /etc/ceph/ceph.keyring - exec: client.0: - ceph_test_libcephsqlite --id libcephsqlite --no-log-to-stderr - workunit: clients: client.0: - rados/test_libcephsqlite.sh cephsqlite env: CEPH_ARGS: --id libcephsqlite --no-log-to-stderr teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_00:43:58 tube: smithi user: kchai verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-08T03:56:46.131 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_5ccf1c30cf724d5371f55439b3b3a90cb994fae2/qa; will attempt to use it 2024-05-08T03:56:46.132 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_5ccf1c30cf724d5371f55439b3b3a90cb994fae2/qa/tasks 2024-05-08T03:56:46.132 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-08T03:56:46.133 INFO:teuthology.task.internal:Checking packages... 2024-05-08T03:56:46.156 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '5ccf1c30cf724d5371f55439b3b3a90cb994fae2' 2024-05-08T03:56:46.156 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-08T03:56:46.156 INFO:teuthology.packaging:ref: None 2024-05-08T03:56:46.157 INFO:teuthology.packaging:tag: None 2024-05-08T03:56:46.157 INFO:teuthology.packaging:branch: wip-kefu-testing-2024-05-06 2024-05-08T03:56:46.157 INFO:teuthology.packaging:sha1: 5ccf1c30cf724d5371f55439b3b3a90cb994fae2 2024-05-08T03:56:46.157 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-kefu-testing-2024-05-06 2024-05-08T03:56:46.376 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3512.g5ccf1c30 2024-05-08T03:56:46.377 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-08T03:56:46.384 INFO:teuthology.task.internal:no buildpackages task found 2024-05-08T03:56:46.384 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-08T03:56:46.421 INFO:teuthology.task.internal:Saving configuration 2024-05-08T03:56:46.433 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-08T03:56:46.445 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T03:56:46.469 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi170.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/kchai-2024-05-08_00:43:58-rados-wip-kefu-testing-2024-05-06-distro-default-smithi/7696821', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 03:51:42.950945', 'locked_by': 'scheduled_kchai@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDISdccAMpjTTpR17SreXiq5NUAYaHMBL61dw0VrbGVPmisU3BYRNBac0cDFl5tw2vHmlDZ77nLl7Dvsv2O1tW8='} 2024-05-08T03:56:46.491 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi187.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/kchai-2024-05-08_00:43:58-rados-wip-kefu-testing-2024-05-06-distro-default-smithi/7696821', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 03:51:42.952483', 'locked_by': 'scheduled_kchai@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA9FW+CA8Pfrhp6UvcPN2OnV1xZvW3bofLOIQf+CZelvy60JgzQQ8NVquLstj4GVp5uSLomwvG5Qy1KSQFa3+ig='} 2024-05-08T03:56:46.491 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-08T03:56:46.499 INFO:teuthology.task.internal:roles: ubuntu@smithi170.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-05-08T03:56:46.499 INFO:teuthology.task.internal:roles: ubuntu@smithi187.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-05-08T03:56:46.499 INFO:teuthology.run_tasks:Running task console_log... 2024-05-08T03:56:46.607 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f13c43d8160>, signals=[15]) 2024-05-08T03:56:46.608 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-08T03:56:46.619 INFO:teuthology.task.internal:Opening connections... 2024-05-08T03:56:46.619 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi170.front.sepia.ceph.com 2024-05-08T03:56:46.621 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T03:56:46.696 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi187.front.sepia.ceph.com 2024-05-08T03:56:46.696 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T03:56:46.768 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-08T03:56:46.781 DEBUG:teuthology.orchestra.run.smithi170:> uname -m 2024-05-08T03:56:46.801 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64 2024-05-08T03:56:46.801 DEBUG:teuthology.orchestra.run.smithi170:> cat /etc/os-release 2024-05-08T03:56:46.858 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="CentOS Stream" 2024-05-08T03:56:46.858 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="9" 2024-05-08T03:56:46.858 INFO:teuthology.orchestra.run.smithi170.stdout:ID="centos" 2024-05-08T03:56:46.858 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE="rhel fedora" 2024-05-08T03:56:46.858 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="9" 2024-05-08T03:56:46.858 INFO:teuthology.orchestra.run.smithi170.stdout:PLATFORM_ID="platform:el9" 2024-05-08T03:56:46.858 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T03:56:46.859 INFO:teuthology.orchestra.run.smithi170.stdout:ANSI_COLOR="0;31" 2024-05-08T03:56:46.859 INFO:teuthology.orchestra.run.smithi170.stdout:LOGO="fedora-logo-icon" 2024-05-08T03:56:46.859 INFO:teuthology.orchestra.run.smithi170.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T03:56:46.859 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://centos.org/" 2024-05-08T03:56:46.859 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T03:56:46.859 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T03:56:46.859 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T03:56:46.859 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2024-05-08T03:56:46.883 DEBUG:teuthology.orchestra.run.smithi187:> uname -m 2024-05-08T03:56:46.901 INFO:teuthology.orchestra.run.smithi187.stdout:x86_64 2024-05-08T03:56:46.901 DEBUG:teuthology.orchestra.run.smithi187:> cat /etc/os-release 2024-05-08T03:56:46.958 INFO:teuthology.orchestra.run.smithi187.stdout:NAME="CentOS Stream" 2024-05-08T03:56:46.958 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION="9" 2024-05-08T03:56:46.959 INFO:teuthology.orchestra.run.smithi187.stdout:ID="centos" 2024-05-08T03:56:46.959 INFO:teuthology.orchestra.run.smithi187.stdout:ID_LIKE="rhel fedora" 2024-05-08T03:56:46.959 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION_ID="9" 2024-05-08T03:56:46.959 INFO:teuthology.orchestra.run.smithi187.stdout:PLATFORM_ID="platform:el9" 2024-05-08T03:56:46.959 INFO:teuthology.orchestra.run.smithi187.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T03:56:46.959 INFO:teuthology.orchestra.run.smithi187.stdout:ANSI_COLOR="0;31" 2024-05-08T03:56:46.959 INFO:teuthology.orchestra.run.smithi187.stdout:LOGO="fedora-logo-icon" 2024-05-08T03:56:46.959 INFO:teuthology.orchestra.run.smithi187.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T03:56:46.959 INFO:teuthology.orchestra.run.smithi187.stdout:HOME_URL="https://centos.org/" 2024-05-08T03:56:46.959 INFO:teuthology.orchestra.run.smithi187.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T03:56:46.959 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T03:56:46.959 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T03:56:46.959 INFO:teuthology.lock.ops:Updating smithi187.front.sepia.ceph.com on lock server 2024-05-08T03:56:46.983 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-08T03:56:46.997 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-08T03:56:47.010 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-08T03:56:47.011 DEBUG:teuthology.orchestra.run.smithi170:> test '!' -e /home/ubuntu/cephtest 2024-05-08T03:56:47.013 DEBUG:teuthology.orchestra.run.smithi187:> test '!' -e /home/ubuntu/cephtest 2024-05-08T03:56:47.029 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-08T03:56:47.040 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-08T03:56:47.040 DEBUG:teuthology.orchestra.run.smithi170:> test -z $(ls -A /var/lib/ceph) 2024-05-08T03:56:47.069 DEBUG:teuthology.orchestra.run.smithi187:> test -z $(ls -A /var/lib/ceph) 2024-05-08T03:56:47.101 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-08T03:56:47.158 INFO:teuthology.run_tasks:Running task kernel... 2024-05-08T03:56:47.174 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-05-08T03:56:47.174 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-05-08T03:56:47.174 DEBUG:teuthology.orchestra.run.smithi170:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T03:56:47.174 DEBUG:teuthology.orchestra.run.smithi187:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T03:56:47.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T03:56:47.191 DEBUG:teuthology.orchestra.run.smithi170:> uname -r 2024-05-08T03:56:47.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T03:56:47.191 DEBUG:teuthology.orchestra.run.smithi187:> uname -r 2024-05-08T03:56:47.249 INFO:teuthology.orchestra.run.smithi187.stdout:5.14.0-437.el9.x86_64 2024-05-08T03:56:47.249 INFO:teuthology.orchestra.run.smithi170.stdout:5.14.0-437.el9.x86_64 2024-05-08T03:56:47.249 INFO:teuthology.task.kernel:Running kernel on smithi187: 5.14.0-437.el9.x86_64 2024-05-08T03:56:47.249 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum install -y kernel 2024-05-08T03:56:47.250 INFO:teuthology.task.kernel:Running kernel on smithi170: 5.14.0-437.el9.x86_64 2024-05-08T03:56:47.250 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum install -y kernel 2024-05-08T03:56:50.935 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - BaseOS 3.9 MB/s | 8.1 MB 00:02 2024-05-08T03:56:53.187 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - BaseOS 1.9 MB/s | 8.1 MB 00:04 2024-05-08T03:56:57.012 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - AppStream 7.0 MB/s | 19 MB 00:02 2024-05-08T03:57:01.038 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-05-08T03:57:02.766 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 16 kB 00:00 2024-05-08T03:57:05.273 INFO:teuthology.orchestra.run.smithi187.stdout:Extra Packages for Enterprise Linux 8.9 MB/s | 21 MB 00:02 2024-05-08T03:57:07.792 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:15 2024-05-08T03:57:10.489 INFO:teuthology.orchestra.run.smithi187.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-05-08T03:57:11.967 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.2 MB 00:00 2024-05-08T03:57:12.512 INFO:teuthology.orchestra.run.smithi187.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-08T03:57:12.513 INFO:teuthology.orchestra.run.smithi187.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-08T03:57:12.587 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2024-05-08T03:57:12.593 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2024-05-08T03:57:12.593 INFO:teuthology.orchestra.run.smithi187.stdout: Package Architecture Version Repository Size 2024-05-08T03:57:12.593 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2024-05-08T03:57:12.593 INFO:teuthology.orchestra.run.smithi187.stdout:Installing: 2024-05-08T03:57:12.593 INFO:teuthology.orchestra.run.smithi187.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-08T03:57:12.593 INFO:teuthology.orchestra.run.smithi187.stdout:Installing dependencies: 2024-05-08T03:57:12.593 INFO:teuthology.orchestra.run.smithi187.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-08T03:57:12.594 INFO:teuthology.orchestra.run.smithi187.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-08T03:57:12.594 INFO:teuthology.orchestra.run.smithi187.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-08T03:57:12.594 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T03:57:12.594 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2024-05-08T03:57:12.594 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2024-05-08T03:57:12.594 INFO:teuthology.orchestra.run.smithi187.stdout:Install 4 Packages 2024-05-08T03:57:12.594 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T03:57:12.594 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 78 M 2024-05-08T03:57:12.594 INFO:teuthology.orchestra.run.smithi187.stdout:Installed size: 123 M 2024-05-08T03:57:12.594 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2024-05-08T03:57:13.577 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 16 kB 00:00 2024-05-08T03:57:13.982 INFO:teuthology.orchestra.run.smithi187.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 359 kB/s | 484 kB 00:01 2024-05-08T03:57:14.778 INFO:teuthology.orchestra.run.smithi170.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-05-08T03:57:17.816 INFO:teuthology.orchestra.run.smithi187.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 3.0 MB/s | 15 MB 00:05 2024-05-08T03:57:19.683 INFO:teuthology.orchestra.run.smithi187.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 4.8 MB/s | 34 MB 00:07 2024-05-08T03:57:20.011 INFO:teuthology.orchestra.run.smithi170.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-05-08T03:57:20.824 INFO:teuthology.orchestra.run.smithi187.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 4.1 MB/s | 28 MB 00:06 2024-05-08T03:57:20.824 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:57:20.824 INFO:teuthology.orchestra.run.smithi187.stdout:Total 9.5 MB/s | 78 MB 00:08 2024-05-08T03:57:21.502 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2024-05-08T03:57:21.834 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2024-05-08T03:57:21.834 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2024-05-08T03:57:22.001 INFO:teuthology.orchestra.run.smithi170.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-08T03:57:22.001 INFO:teuthology.orchestra.run.smithi170.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-08T03:57:22.107 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-05-08T03:57:22.116 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-05-08T03:57:22.116 INFO:teuthology.orchestra.run.smithi170.stdout: Package Architecture Version Repository Size 2024-05-08T03:57:22.116 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-05-08T03:57:22.116 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2024-05-08T03:57:22.116 INFO:teuthology.orchestra.run.smithi170.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-08T03:57:22.116 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2024-05-08T03:57:22.116 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-08T03:57:22.116 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-08T03:57:22.116 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-08T03:57:22.116 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T03:57:22.116 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-05-08T03:57:22.116 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-05-08T03:57:22.117 INFO:teuthology.orchestra.run.smithi170.stdout:Install 4 Packages 2024-05-08T03:57:22.117 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T03:57:22.117 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 78 M 2024-05-08T03:57:22.117 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 123 M 2024-05-08T03:57:22.117 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2024-05-08T03:57:22.752 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2024-05-08T03:57:22.752 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2024-05-08T03:57:23.188 INFO:teuthology.orchestra.run.smithi170.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 520 kB/s | 484 kB 00:00 2024-05-08T03:57:23.913 INFO:teuthology.orchestra.run.smithi170.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 9.3 MB/s | 15 MB 00:01 2024-05-08T03:57:24.713 INFO:teuthology.orchestra.run.smithi170.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 14 MB/s | 34 MB 00:02 2024-05-08T03:57:25.187 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2024-05-08T03:57:25.813 INFO:teuthology.orchestra.run.smithi170.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 11 MB/s | 28 MB 00:02 2024-05-08T03:57:25.814 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2024-05-08T03:57:25.814 INFO:teuthology.orchestra.run.smithi170.stdout:Total 21 MB/s | 78 MB 00:03 2024-05-08T03:57:25.882 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-08T03:57:25.965 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T03:57:26.490 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2024-05-08T03:57:26.822 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2024-05-08T03:57:26.822 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2024-05-08T03:57:27.755 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2024-05-08T03:57:27.755 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2024-05-08T03:57:28.643 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T03:57:28.767 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T03:57:30.167 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2024-05-08T03:57:30.903 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-08T03:57:30.997 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T03:57:33.473 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T03:57:33.511 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T03:57:33.682 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T03:57:33.827 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T03:57:38.234 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T03:57:38.540 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T03:57:38.755 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T03:57:43.450 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T03:58:07.430 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T03:58:07.468 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-08T03:58:07.771 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T03:58:07.771 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-08T03:58:07.771 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T03:58:07.771 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T03:58:08.563 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T03:58:08.563 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T03:58:08.563 INFO:teuthology.orchestra.run.smithi187.stdout:Installed: 2024-05-08T03:58:08.563 INFO:teuthology.orchestra.run.smithi187.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-08T03:58:08.563 INFO:teuthology.orchestra.run.smithi187.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-08T03:58:08.563 INFO:teuthology.orchestra.run.smithi187.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-08T03:58:08.563 INFO:teuthology.orchestra.run.smithi187.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-08T03:58:08.563 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T03:58:08.563 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2024-05-08T03:58:08.915 DEBUG:teuthology.orchestra.run.smithi187:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T03:58:08.987 INFO:teuthology.orchestra.run.smithi187.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T03:58:08.987 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T03:58:08.988 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-08T03:58:08.988 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-08T03:58:08.988 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-08T03:58:08.988 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-05-08T03:58:08.988 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi187.front.sepia.ceph.com, path=None, version=distro) 2024-05-08T03:58:08.988 DEBUG:teuthology.orchestra.run.smithi187:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T03:58:09.037 INFO:teuthology.orchestra.run.smithi187.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T03:58:09.038 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T03:58:09.038 DEBUG:teuthology.orchestra.run.smithi187:> sudo rpm -qi grub2-tools 2024-05-08T03:58:09.071 INFO:teuthology.orchestra.run.smithi187.stdout:Name : grub2-tools 2024-05-08T03:58:09.071 INFO:teuthology.orchestra.run.smithi187.stdout:Epoch : 1 2024-05-08T03:58:09.071 INFO:teuthology.orchestra.run.smithi187.stdout:Version : 2.06 2024-05-08T03:58:09.071 INFO:teuthology.orchestra.run.smithi187.stdout:Release : 68.el9 2024-05-08T03:58:09.071 INFO:teuthology.orchestra.run.smithi187.stdout:Architecture: x86_64 2024-05-08T03:58:09.072 INFO:teuthology.orchestra.run.smithi187.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-08T03:58:09.072 INFO:teuthology.orchestra.run.smithi187.stdout:Group : Unspecified 2024-05-08T03:58:09.072 INFO:teuthology.orchestra.run.smithi187.stdout:Size : 8264254 2024-05-08T03:58:09.072 INFO:teuthology.orchestra.run.smithi187.stdout:License : GPLv3+ 2024-05-08T03:58:09.072 INFO:teuthology.orchestra.run.smithi187.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-08T03:58:09.072 INFO:teuthology.orchestra.run.smithi187.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-08T03:58:09.072 INFO:teuthology.orchestra.run.smithi187.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-08T03:58:09.072 INFO:teuthology.orchestra.run.smithi187.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-08T03:58:09.072 INFO:teuthology.orchestra.run.smithi187.stdout:Packager : builder@centos.org 2024-05-08T03:58:09.072 INFO:teuthology.orchestra.run.smithi187.stdout:Vendor : CentOS 2024-05-08T03:58:09.072 INFO:teuthology.orchestra.run.smithi187.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-08T03:58:09.072 INFO:teuthology.orchestra.run.smithi187.stdout:Summary : Support tools for GRUB. 2024-05-08T03:58:09.072 INFO:teuthology.orchestra.run.smithi187.stdout:Description : 2024-05-08T03:58:09.072 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T03:58:09.073 INFO:teuthology.orchestra.run.smithi187.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-08T03:58:09.073 INFO:teuthology.orchestra.run.smithi187.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-08T03:58:09.073 INFO:teuthology.orchestra.run.smithi187.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-08T03:58:09.073 INFO:teuthology.orchestra.run.smithi187.stdout:hardware devices. 2024-05-08T03:58:09.073 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T03:58:09.073 INFO:teuthology.orchestra.run.smithi187.stdout:This subpackage provides tools for support of all platforms. 2024-05-08T03:58:09.074 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-08T03:58:09.074 INFO:teuthology.task.kernel:Updating grub on smithi187 to boot 5.14.0-444.el9.x86_64 2024-05-08T03:58:09.074 DEBUG:teuthology.orchestra.run.smithi187:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-08T03:58:09.708 INFO:teuthology.orchestra.run.smithi187.stderr:Generating grub configuration file ... 2024-05-08T03:58:11.173 INFO:teuthology.orchestra.run.smithi187.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-08T03:58:11.199 INFO:teuthology.orchestra.run.smithi187.stderr:done 2024-05-08T03:58:11.201 DEBUG:teuthology.orchestra.run.smithi187:> mktemp 2024-05-08T03:58:11.215 INFO:teuthology.orchestra.run.smithi187.stdout:/tmp/tmp.jBB4V8u0LU 2024-05-08T03:58:11.215 DEBUG:teuthology.orchestra.run.smithi187:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.jBB4V8u0LU 2024-05-08T03:58:11.282 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0666 /tmp/tmp.jBB4V8u0LU 2024-05-08T03:58:11.420 DEBUG:teuthology.orchestra.remote:smithi187:/tmp/tmp.jBB4V8u0LU is 6KB 2024-05-08T03:58:11.467 DEBUG:teuthology.orchestra.run.smithi187:> rm -fr /tmp/tmp.jBB4V8u0LU 2024-05-08T03:58:11.481 DEBUG:teuthology.orchestra.run.smithi187:> sudo /bin/ls /boot/loader/entries || true 2024-05-08T03:58:11.542 INFO:teuthology.orchestra.run.smithi187.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-08T03:58:11.543 INFO:teuthology.orchestra.run.smithi187.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-08T03:58:11.543 INFO:teuthology.orchestra.run.smithi187.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-08T03:58:11.543 INFO:teuthology.orchestra.run.smithi187.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-08T03:58:11.544 DEBUG:teuthology.orchestra.run.smithi187:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-08T03:58:11.697 DEBUG:teuthology.orchestra.run.smithi187:> sudo shutdown -r now 2024-05-08T03:58:12.709 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T03:58:12.746 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-08T03:58:13.076 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T03:58:13.076 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-08T03:58:13.076 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T03:58:13.076 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T03:58:13.867 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T03:58:13.867 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T03:58:13.868 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2024-05-08T03:58:13.868 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-08T03:58:13.868 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-08T03:58:13.868 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-08T03:58:13.868 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-08T03:58:13.868 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T03:58:13.868 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-05-08T03:58:14.228 DEBUG:teuthology.orchestra.run.smithi170:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T03:58:14.300 INFO:teuthology.orchestra.run.smithi170.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T03:58:14.301 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T03:58:14.301 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-08T03:58:14.301 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-08T03:58:14.301 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-08T03:58:14.301 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-05-08T03:58:14.301 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi170.front.sepia.ceph.com, path=None, version=distro) 2024-05-08T03:58:14.301 DEBUG:teuthology.orchestra.run.smithi170:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T03:58:14.350 INFO:teuthology.orchestra.run.smithi170.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T03:58:14.351 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T03:58:14.351 DEBUG:teuthology.orchestra.run.smithi170:> sudo rpm -qi grub2-tools 2024-05-08T03:58:14.381 INFO:teuthology.orchestra.run.smithi170.stdout:Name : grub2-tools 2024-05-08T03:58:14.381 INFO:teuthology.orchestra.run.smithi170.stdout:Epoch : 1 2024-05-08T03:58:14.381 INFO:teuthology.orchestra.run.smithi170.stdout:Version : 2.06 2024-05-08T03:58:14.382 INFO:teuthology.orchestra.run.smithi170.stdout:Release : 68.el9 2024-05-08T03:58:14.382 INFO:teuthology.orchestra.run.smithi170.stdout:Architecture: x86_64 2024-05-08T03:58:14.382 INFO:teuthology.orchestra.run.smithi170.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-08T03:58:14.382 INFO:teuthology.orchestra.run.smithi170.stdout:Group : Unspecified 2024-05-08T03:58:14.382 INFO:teuthology.orchestra.run.smithi170.stdout:Size : 8264254 2024-05-08T03:58:14.382 INFO:teuthology.orchestra.run.smithi170.stdout:License : GPLv3+ 2024-05-08T03:58:14.382 INFO:teuthology.orchestra.run.smithi170.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-08T03:58:14.382 INFO:teuthology.orchestra.run.smithi170.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-08T03:58:14.382 INFO:teuthology.orchestra.run.smithi170.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-08T03:58:14.382 INFO:teuthology.orchestra.run.smithi170.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-08T03:58:14.382 INFO:teuthology.orchestra.run.smithi170.stdout:Packager : builder@centos.org 2024-05-08T03:58:14.382 INFO:teuthology.orchestra.run.smithi170.stdout:Vendor : CentOS 2024-05-08T03:58:14.382 INFO:teuthology.orchestra.run.smithi170.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-08T03:58:14.382 INFO:teuthology.orchestra.run.smithi170.stdout:Summary : Support tools for GRUB. 2024-05-08T03:58:14.383 INFO:teuthology.orchestra.run.smithi170.stdout:Description : 2024-05-08T03:58:14.383 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T03:58:14.383 INFO:teuthology.orchestra.run.smithi170.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-08T03:58:14.383 INFO:teuthology.orchestra.run.smithi170.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-08T03:58:14.383 INFO:teuthology.orchestra.run.smithi170.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-08T03:58:14.383 INFO:teuthology.orchestra.run.smithi170.stdout:hardware devices. 2024-05-08T03:58:14.383 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T03:58:14.383 INFO:teuthology.orchestra.run.smithi170.stdout:This subpackage provides tools for support of all platforms. 2024-05-08T03:58:14.384 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-08T03:58:14.384 INFO:teuthology.task.kernel:Updating grub on smithi170 to boot 5.14.0-444.el9.x86_64 2024-05-08T03:58:14.384 DEBUG:teuthology.orchestra.run.smithi170:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-08T03:58:15.005 INFO:teuthology.orchestra.run.smithi170.stderr:Generating grub configuration file ... 2024-05-08T03:58:16.409 INFO:teuthology.orchestra.run.smithi170.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-08T03:58:16.437 INFO:teuthology.orchestra.run.smithi170.stderr:done 2024-05-08T03:58:16.439 DEBUG:teuthology.orchestra.run.smithi170:> mktemp 2024-05-08T03:58:16.456 INFO:teuthology.orchestra.run.smithi170.stdout:/tmp/tmp.kEXDKldA5s 2024-05-08T03:58:16.456 DEBUG:teuthology.orchestra.run.smithi170:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.kEXDKldA5s 2024-05-08T03:58:16.521 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0666 /tmp/tmp.kEXDKldA5s 2024-05-08T03:58:16.659 DEBUG:teuthology.orchestra.remote:smithi170:/tmp/tmp.kEXDKldA5s is 6KB 2024-05-08T03:58:16.706 DEBUG:teuthology.orchestra.run.smithi170:> rm -fr /tmp/tmp.kEXDKldA5s 2024-05-08T03:58:16.720 DEBUG:teuthology.orchestra.run.smithi170:> sudo /bin/ls /boot/loader/entries || true 2024-05-08T03:58:16.782 INFO:teuthology.orchestra.run.smithi170.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-08T03:58:16.782 INFO:teuthology.orchestra.run.smithi170.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-08T03:58:16.782 INFO:teuthology.orchestra.run.smithi170.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-08T03:58:16.782 INFO:teuthology.orchestra.run.smithi170.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-08T03:58:16.784 DEBUG:teuthology.orchestra.run.smithi170:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-08T03:58:16.926 DEBUG:teuthology.orchestra.run.smithi170:> sudo shutdown -r now 2024-05-08T03:58:41.722 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-08T03:58:41.723 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-05-08T03:58:41.724 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T03:58:46.930 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-08T03:58:46.931 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-05-08T03:58:46.931 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T03:59:00.131 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.187 2024-05-08T03:59:05.250 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.170 2024-05-08T03:59:09.135 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-05-08T03:59:09.136 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T03:59:12.190 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.187 2024-05-08T03:59:14.254 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-05-08T03:59:14.254 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T03:59:24.198 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-05-08T03:59:24.199 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T03:59:42.627 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.187 2024-05-08T03:59:57.642 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-05-08T03:59:57.644 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T03:59:58.333 DEBUG:teuthology.orchestra.run.smithi187:> true 2024-05-08T03:59:58.811 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-05-08T03:59:58.812 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-444.el9.x86_64"... 2024-05-08T03:59:58.812 DEBUG:teuthology.orchestra.run.smithi187:> uname -r 2024-05-08T03:59:58.837 INFO:teuthology.orchestra.run.smithi187.stdout:5.14.0-444.el9.x86_64 2024-05-08T03:59:58.837 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-08T03:59:58.837 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-08T03:59:58.838 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-08T03:59:59.839 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-08T03:59:59.839 DEBUG:teuthology.orchestra.run.smithi187:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-08T03:59:59.960 INFO:teuthology.orchestra.run.smithi187.stdout:ttyS1 2024-05-08T04:00:00.195 DEBUG:teuthology.parallel:result is None 2024-05-08T04:00:14.257 DEBUG:teuthology.orchestra.remote:timed out 2024-05-08T04:00:26.258 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-05-08T04:00:26.259 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T04:00:26.649 DEBUG:teuthology.orchestra.run.smithi170:> true 2024-05-08T04:00:27.032 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-05-08T04:00:27.032 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-444.el9.x86_64"... 2024-05-08T04:00:27.032 DEBUG:teuthology.orchestra.run.smithi170:> uname -r 2024-05-08T04:00:27.048 INFO:teuthology.orchestra.run.smithi170.stdout:5.14.0-444.el9.x86_64 2024-05-08T04:00:27.048 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-08T04:00:27.048 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-08T04:00:27.049 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-08T04:00:28.050 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-08T04:00:28.050 DEBUG:teuthology.orchestra.run.smithi170:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-08T04:00:28.114 INFO:teuthology.orchestra.run.smithi170.stdout:ttyS1 2024-05-08T04:00:28.147 DEBUG:teuthology.parallel:result is None 2024-05-08T04:00:28.147 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-08T04:00:28.154 INFO:teuthology.task.internal:Creating test directory... 2024-05-08T04:00:28.155 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T04:00:28.158 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T04:00:28.176 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-08T04:00:28.183 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-08T04:00:28.190 INFO:teuthology.task.internal:Creating archive directory... 2024-05-08T04:00:28.190 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T04:00:28.215 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T04:00:28.299 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-08T04:00:28.306 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-08T04:00:28.306 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T04:00:28.309 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T04:00:28.338 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T04:00:28.344 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T04:00:28.348 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T04:00:28.356 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T04:00:28.357 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-08T04:00:28.364 INFO:teuthology.task.internal:Configuring sudo... 2024-05-08T04:00:28.365 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T04:00:28.393 DEBUG:teuthology.orchestra.run.smithi187:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T04:00:28.425 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-08T04:00:28.433 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-08T04:00:28.434 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T04:00:28.460 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T04:00:28.481 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T04:00:28.556 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T04:00:28.607 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T04:00:28.690 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T04:00:28.730 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-08T04:00:28.730 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T04:00:28.795 DEBUG:teuthology.orchestra.run.smithi187:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T04:00:28.842 DEBUG:teuthology.orchestra.run.smithi187:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T04:00:28.894 DEBUG:teuthology.orchestra.run.smithi187:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T04:00:28.975 DEBUG:teuthology.orchestra.run.smithi187:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T04:00:29.019 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-08T04:00:29.020 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T04:00:29.086 DEBUG:teuthology.orchestra.run.smithi170:> sudo service rsyslog restart 2024-05-08T04:00:29.089 DEBUG:teuthology.orchestra.run.smithi187:> sudo service rsyslog restart 2024-05-08T04:00:29.129 INFO:teuthology.orchestra.run.smithi170.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T04:00:29.173 INFO:teuthology.orchestra.run.smithi187.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T04:00:29.552 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-08T04:00:29.560 INFO:teuthology.task.internal:Starting timer... 2024-05-08T04:00:29.560 INFO:teuthology.run_tasks:Running task pcp... 2024-05-08T04:00:29.573 INFO:teuthology.run_tasks:Running task selinux... 2024-05-08T04:00:29.582 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-05-08T04:00:29.583 DEBUG:teuthology.orchestra.run.smithi170:> sudo service auditd rotate 2024-05-08T04:00:29.651 INFO:teuthology.orchestra.run.smithi170.stdout:Rotating logs: 2024-05-08T04:00:29.652 DEBUG:teuthology.orchestra.run.smithi187:> sudo service auditd rotate 2024-05-08T04:00:29.700 INFO:teuthology.orchestra.run.smithi187.stdout:Rotating logs: 2024-05-08T04:00:29.702 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-08T04:00:29.702 DEBUG:teuthology.orchestra.run.smithi170:> /usr/sbin/getenforce 2024-05-08T04:00:29.748 INFO:teuthology.orchestra.run.smithi170.stdout:Permissive 2024-05-08T04:00:29.749 DEBUG:teuthology.orchestra.run.smithi187:> /usr/sbin/getenforce 2024-05-08T04:00:29.773 INFO:teuthology.orchestra.run.smithi187.stdout:Permissive 2024-05-08T04:00:29.774 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi170.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi187.front.sepia.ceph.com': 'permissive'} 2024-05-08T04:00:29.774 DEBUG:teuthology.orchestra.run.smithi170:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T04:00:29.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T04:00:29.801 DEBUG:teuthology.orchestra.run.smithi187:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T04:00:29.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T04:00:29.841 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-08T04:00:29.841 DEBUG:teuthology.orchestra.run.smithi170:> sudo /usr/sbin/setenforce permissive 2024-05-08T04:00:29.868 DEBUG:teuthology.orchestra.run.smithi187:> sudo /usr/sbin/setenforce permissive 2024-05-08T04:00:29.917 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-08T04:00:29.928 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-08T04:00:29.944 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-08T04:00:29.945 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi170.front.sepia.ceph.com,smithi187.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-08T04:05:35.576 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi170.front.sepia.ceph.com'), Remote(name='ubuntu@smithi187.front.sepia.ceph.com')] 2024-05-08T04:05:35.578 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-05-08T04:05:35.580 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T04:05:35.656 DEBUG:teuthology.orchestra.run.smithi170:> true 2024-05-08T04:05:35.745 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-05-08T04:05:35.745 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-05-08T04:05:35.746 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T04:05:35.861 DEBUG:teuthology.orchestra.run.smithi187:> true 2024-05-08T04:05:35.943 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-05-08T04:05:35.943 INFO:teuthology.run_tasks:Running task clock... 2024-05-08T04:05:35.954 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-08T04:05:35.954 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T04:05:35.955 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T04:05:35.957 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T04:05:35.957 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T04:05:35.985 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T04:05:35.999 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T04:05:36.012 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T04:05:36.023 INFO:teuthology.orchestra.run.smithi170.stderr:sudo: ntpd: command not found 2024-05-08T04:05:36.026 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T04:05:36.034 INFO:teuthology.orchestra.run.smithi170.stdout:506 Cannot talk to daemon 2024-05-08T04:05:36.048 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T04:05:36.051 INFO:teuthology.orchestra.run.smithi187.stderr:sudo: ntpd: command not found 2024-05-08T04:05:36.062 INFO:teuthology.orchestra.run.smithi187.stdout:506 Cannot talk to daemon 2024-05-08T04:05:36.062 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T04:05:36.075 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T04:05:36.089 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T04:05:36.117 INFO:teuthology.orchestra.run.smithi170.stderr:bash: line 1: ntpq: command not found 2024-05-08T04:05:36.120 INFO:teuthology.orchestra.run.smithi170.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T04:05:36.120 INFO:teuthology.orchestra.run.smithi170.stdout:=============================================================================== 2024-05-08T04:05:36.120 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T04:05:36.120 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T04:05:36.120 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T04:05:36.120 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T04:05:36.140 INFO:teuthology.orchestra.run.smithi187.stderr:bash: line 1: ntpq: command not found 2024-05-08T04:05:36.144 INFO:teuthology.orchestra.run.smithi187.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T04:05:36.144 INFO:teuthology.orchestra.run.smithi187.stdout:=============================================================================== 2024-05-08T04:05:36.145 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T04:05:36.145 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T04:05:36.145 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T04:05:36.145 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T04:05:36.145 INFO:teuthology.run_tasks:Running task install... 2024-05-08T04:05:36.155 DEBUG:teuthology.task.install:project ceph 2024-05-08T04:05:36.155 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '5ccf1c30cf724d5371f55439b3b3a90cb994fae2'}} 2024-05-08T04:05:36.156 DEBUG:teuthology.task.install:config {'extra_system_packages': {'deb': ['sqlite3'], 'rpm': ['sqlite-devel']}, 'flavor': 'default', 'sha1': '5ccf1c30cf724d5371f55439b3b3a90cb994fae2'} 2024-05-08T04:05:36.156 INFO:teuthology.task.install:Using flavor: default 2024-05-08T04:05:36.162 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-08T04:05:36.162 INFO:teuthology.task.install:extra packages: [] 2024-05-08T04:05:36.162 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': {'deb': ['sqlite3'], 'rpm': ['sqlite-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5ccf1c30cf724d5371f55439b3b3a90cb994fae2', 'tag': None, 'wait_for_package': False} 2024-05-08T04:05:36.163 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5ccf1c30cf724d5371f55439b3b3a90cb994fae2 2024-05-08T04:05:36.165 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': {'deb': ['sqlite3'], 'rpm': ['sqlite-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5ccf1c30cf724d5371f55439b3b3a90cb994fae2', 'tag': None, 'wait_for_package': False} 2024-05-08T04:05:36.165 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5ccf1c30cf724d5371f55439b3b3a90cb994fae2 2024-05-08T04:05:36.316 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-kefu-testing-2024-05-06/5ccf1c30cf724d5371f55439b3b3a90cb994fae2/centos/9/flavors/default/ 2024-05-08T04:05:36.316 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3512.g5ccf1c30 2024-05-08T04:05:36.431 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-kefu-testing-2024-05-06/5ccf1c30cf724d5371f55439b3b3a90cb994fae2/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-kefu-testing-2024-05-06/5ccf1c30cf724d5371f55439b3b3a90cb994fae2/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-kefu-testing-2024-05-06/5ccf1c30cf724d5371f55439b3b3a90cb994fae2/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T04:05:36.432 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-08T04:05:36.432 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T04:05:36.435 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-kefu-testing-2024-05-06/5ccf1c30cf724d5371f55439b3b3a90cb994fae2/centos/9/flavors/default/ 2024-05-08T04:05:36.436 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3512.g5ccf1c30 2024-05-08T04:05:36.458 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, sqlite-devel, sqlite-devel, sqlite-devel on remote rpm x86_64 2024-05-08T04:05:36.458 DEBUG:teuthology.orchestra.run.smithi170:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5ccf1c30cf724d5371f55439b3b3a90cb994fae2/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T04:05:36.524 DEBUG:teuthology.orchestra.run.smithi170:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T04:05:36.552 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-kefu-testing-2024-05-06/5ccf1c30cf724d5371f55439b3b3a90cb994fae2/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-kefu-testing-2024-05-06/5ccf1c30cf724d5371f55439b3b3a90cb994fae2/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-kefu-testing-2024-05-06/5ccf1c30cf724d5371f55439b3b3a90cb994fae2/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T04:05:36.552 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-08T04:05:36.552 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T04:05:36.580 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, sqlite-devel, sqlite-devel, sqlite-devel, sqlite-devel on remote rpm x86_64 2024-05-08T04:05:36.580 DEBUG:teuthology.orchestra.run.smithi187:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5ccf1c30cf724d5371f55439b3b3a90cb994fae2/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T04:05:36.599 DEBUG:teuthology.orchestra.run.smithi170:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T04:05:36.646 DEBUG:teuthology.orchestra.run.smithi187:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T04:05:36.663 INFO:teuthology.orchestra.run.smithi170.stdout:check_obsoletes = 1 2024-05-08T04:05:36.665 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum clean all 2024-05-08T04:05:36.723 DEBUG:teuthology.orchestra.run.smithi187:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T04:05:36.787 INFO:teuthology.orchestra.run.smithi187.stdout:check_obsoletes = 1 2024-05-08T04:05:36.789 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum clean all 2024-05-08T04:05:36.905 INFO:teuthology.orchestra.run.smithi170.stdout:54 files removed 2024-05-08T04:05:36.933 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd sqlite-devel sqlite-devel sqlite-devel sqlite-devel 2024-05-08T04:05:37.073 INFO:teuthology.orchestra.run.smithi187.stdout:54 files removed 2024-05-08T04:05:37.101 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd sqlite-devel sqlite-devel sqlite-devel sqlite-devel 2024-05-08T04:05:37.752 INFO:teuthology.orchestra.run.smithi170.stdout:ceph packages for x86_64 154 kB/s | 84 kB 00:00 2024-05-08T04:05:37.790 INFO:teuthology.orchestra.run.smithi187.stdout:ceph packages for x86_64 202 kB/s | 84 kB 00:00 2024-05-08T04:05:38.153 INFO:teuthology.orchestra.run.smithi170.stdout:ceph noarch packages 34 kB/s | 12 kB 00:00 2024-05-08T04:05:38.166 INFO:teuthology.orchestra.run.smithi187.stdout:ceph noarch packages 36 kB/s | 12 kB 00:00 2024-05-08T04:05:38.508 INFO:teuthology.orchestra.run.smithi187.stdout:ceph source packages 6.4 kB/s | 1.9 kB 00:00 2024-05-08T04:05:38.578 INFO:teuthology.orchestra.run.smithi170.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-05-08T04:05:39.295 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.1 MB 00:00 2024-05-08T04:05:39.941 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - BaseOS 5.8 MB/s | 8.1 MB 00:01 2024-05-08T04:05:41.096 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-05-08T04:05:43.510 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - AppStream 7.7 MB/s | 19 MB 00:02 2024-05-08T04:05:45.114 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.2 MB 00:00 2024-05-08T04:05:46.807 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 16 kB 00:00 2024-05-08T04:05:48.378 INFO:teuthology.orchestra.run.smithi170.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-05-08T04:05:53.600 INFO:teuthology.orchestra.run.smithi170.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-05-08T04:05:55.926 INFO:teuthology.orchestra.run.smithi170.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T04:05:55.927 INFO:teuthology.orchestra.run.smithi170.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T04:05:55.984 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-05-08T04:05:56.002 INFO:teuthology.orchestra.run.smithi170.stdout:======================================================================================= 2024-05-08T04:05:56.002 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2024-05-08T04:05:56.002 INFO:teuthology.orchestra.run.smithi170.stdout:======================================================================================= 2024-05-08T04:05:56.002 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2024-05-08T04:05:56.002 INFO:teuthology.orchestra.run.smithi170.stdout: ceph x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 6.6 k 2024-05-08T04:05:56.002 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 5.4 M 2024-05-08T04:05:56.002 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 890 k 2024-05-08T04:05:56.002 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 145 k 2024-05-08T04:05:56.002 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 1.5 M 2024-05-08T04:05:56.002 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3512.g5ccf1c30.el9 ceph-noarch 142 k 2024-05-08T04:05:56.003 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3512.g5ccf1c30.el9 ceph-noarch 3.5 M 2024-05-08T04:05:56.003 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3512.g5ccf1c30.el9 ceph-noarch 7.4 M 2024-05-08T04:05:56.003 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook noarch 2:19.0.0-3512.g5ccf1c30.el9 ceph-noarch 49 k 2024-05-08T04:05:56.003 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 11 M 2024-05-08T04:05:56.003 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 49 M 2024-05-08T04:05:56.003 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-volume noarch 2:19.0.0-3512.g5ccf1c30.el9 ceph-noarch 273 k 2024-05-08T04:05:56.003 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm noarch 2:19.0.0-3512.g5ccf1c30.el9 ceph-noarch 764 k 2024-05-08T04:05:56.003 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 32 k 2024-05-08T04:05:56.003 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2 x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 747 k 2024-05-08T04:05:56.003 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 127 k 2024-05-08T04:05:56.003 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 161 k 2024-05-08T04:05:56.003 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 321 k 2024-05-08T04:05:56.003 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 297 k 2024-05-08T04:05:56.003 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 100 k 2024-05-08T04:05:56.004 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 85 k 2024-05-08T04:05:56.004 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 3.1 M 2024-05-08T04:05:56.004 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 170 k 2024-05-08T04:05:56.004 INFO:teuthology.orchestra.run.smithi170.stdout: sqlite-devel x86_64 3.34.1-7.el9 appstream 134 k 2024-05-08T04:05:56.004 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrading: 2024-05-08T04:05:56.004 INFO:teuthology.orchestra.run.smithi170.stdout: librados2 x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 3.4 M 2024-05-08T04:05:56.004 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1 x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 3.1 M 2024-05-08T04:05:56.004 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2024-05-08T04:05:56.004 INFO:teuthology.orchestra.run.smithi170.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T04:05:56.004 INFO:teuthology.orchestra.run.smithi170.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T04:05:56.004 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 21 M 2024-05-08T04:05:56.004 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3512.g5ccf1c30.el9 ceph-noarch 34 k 2024-05-08T04:05:56.004 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 2.4 M 2024-05-08T04:05:56.004 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3512.g5ccf1c30.el9 ceph-noarch 274 k 2024-05-08T04:05:56.004 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 4.7 M 2024-05-08T04:05:56.005 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 17 M 2024-05-08T04:05:56.005 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3512.g5ccf1c30.el9 ceph-noarch 16 k 2024-05-08T04:05:56.005 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 25 k 2024-05-08T04:05:56.005 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T04:05:56.005 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T04:05:56.005 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T04:05:56.005 INFO:teuthology.orchestra.run.smithi170.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T04:05:56.005 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T04:05:56.005 INFO:teuthology.orchestra.run.smithi170.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T04:05:56.005 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T04:05:56.005 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T04:05:56.005 INFO:teuthology.orchestra.run.smithi170.stdout: libcephsqlite x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 164 k 2024-05-08T04:05:56.005 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T04:05:56.005 INFO:teuthology.orchestra.run.smithi170.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T04:05:56.006 INFO:teuthology.orchestra.run.smithi170.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T04:05:56.006 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T04:05:56.006 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T04:05:56.006 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1 x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 500 k 2024-05-08T04:05:56.006 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T04:05:56.006 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2 x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 5.4 M 2024-05-08T04:05:56.006 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T04:05:56.006 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T04:05:56.006 INFO:teuthology.orchestra.run.smithi170.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T04:05:56.006 INFO:teuthology.orchestra.run.smithi170.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T04:05:56.006 INFO:teuthology.orchestra.run.smithi170.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T04:05:56.006 INFO:teuthology.orchestra.run.smithi170.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T04:05:56.006 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T04:05:56.007 INFO:teuthology.orchestra.run.smithi170.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T04:05:56.007 INFO:teuthology.orchestra.run.smithi170.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T04:05:56.007 INFO:teuthology.orchestra.run.smithi170.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T04:05:56.007 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T04:05:56.007 INFO:teuthology.orchestra.run.smithi170.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T04:05:56.007 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T04:05:56.007 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T04:05:56.007 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 45 k 2024-05-08T04:05:56.007 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 135 k 2024-05-08T04:05:56.007 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T04:05:56.007 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T04:05:56.007 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T04:05:56.007 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T04:05:56.007 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T04:05:56.007 INFO:teuthology.orchestra.run.smithi170.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T04:05:56.008 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T04:05:56.008 INFO:teuthology.orchestra.run.smithi170.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T04:05:56.008 INFO:teuthology.orchestra.run.smithi170.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T04:05:56.008 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T04:05:56.008 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T04:05:56.008 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-08T04:05:56.008 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T04:05:56.008 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T04:05:56.008 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T04:05:56.008 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T04:05:56.008 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T04:05:56.008 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T04:05:56.008 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T04:05:56.008 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T04:05:56.009 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T04:05:56.009 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T04:05:56.009 INFO:teuthology.orchestra.run.smithi170.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T04:05:56.009 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T04:05:56.009 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T04:05:56.009 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T04:05:56.009 INFO:teuthology.orchestra.run.smithi170.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T04:05:56.009 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T04:05:56.009 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T04:05:56.009 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T04:05:56.009 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T04:05:56.009 INFO:teuthology.orchestra.run.smithi170.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T04:05:56.009 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T04:05:56.009 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T04:05:56.009 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T04:05:56.010 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T04:05:56.010 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T04:05:56.010 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T04:05:56.010 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T04:05:56.010 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T04:05:56.010 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T04:05:56.010 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T04:05:56.010 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T04:05:56.010 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T04:05:56.010 INFO:teuthology.orchestra.run.smithi170.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T04:05:56.010 INFO:teuthology.orchestra.run.smithi170.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T04:05:56.010 INFO:teuthology.orchestra.run.smithi170.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T04:05:56.010 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T04:05:56.010 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T04:05:56.011 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T04:05:56.011 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T04:05:56.011 INFO:teuthology.orchestra.run.smithi170.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T04:05:56.011 INFO:teuthology.orchestra.run.smithi170.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T04:05:56.011 INFO:teuthology.orchestra.run.smithi170.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T04:05:56.011 INFO:teuthology.orchestra.run.smithi170.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T04:05:56.011 INFO:teuthology.orchestra.run.smithi170.stdout: sqlite x86_64 3.34.1-7.el9 appstream 748 k 2024-05-08T04:05:56.011 INFO:teuthology.orchestra.run.smithi170.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T04:05:56.011 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T04:05:56.011 INFO:teuthology.orchestra.run.smithi170.stdout:Installing weak dependencies: 2024-05-08T04:05:56.011 INFO:teuthology.orchestra.run.smithi170.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T04:05:56.011 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:05:56.011 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-05-08T04:05:56.011 INFO:teuthology.orchestra.run.smithi170.stdout:======================================================================================= 2024-05-08T04:05:56.011 INFO:teuthology.orchestra.run.smithi170.stdout:Install 126 Packages 2024-05-08T04:05:56.012 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrade 2 Packages 2024-05-08T04:05:56.012 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:05:56.012 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 203 M 2024-05-08T04:05:56.012 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2024-05-08T04:05:56.802 INFO:teuthology.orchestra.run.smithi170.stdout:(1/128): ceph-19.0.0-3512.g5ccf1c30.el9.x86_64. 30 kB/s | 6.6 kB 00:00 2024-05-08T04:05:57.094 INFO:teuthology.orchestra.run.smithi170.stdout:(2/128): ceph-fuse-19.0.0-3512.g5ccf1c30.el9.x8 3.0 MB/s | 890 kB 00:00 2024-05-08T04:05:57.219 INFO:teuthology.orchestra.run.smithi170.stdout:(3/128): ceph-immutable-object-cache-19.0.0-351 1.1 MB/s | 145 kB 00:00 2024-05-08T04:05:57.319 INFO:teuthology.orchestra.run.smithi170.stdout:(4/128): ceph-base-19.0.0-3512.g5ccf1c30.el9.x8 7.3 MB/s | 5.4 MB 00:00 2024-05-08T04:05:57.528 INFO:teuthology.orchestra.run.smithi170.stdout:(5/128): ceph-mgr-19.0.0-3512.g5ccf1c30.el9.x86 7.3 MB/s | 1.5 MB 00:00 2024-05-08T04:05:57.620 INFO:teuthology.orchestra.run.smithi170.stdout:(6/128): ceph-mds-19.0.0-3512.g5ccf1c30.el9.x86 5.9 MB/s | 2.4 MB 00:00 2024-05-08T04:05:57.937 INFO:teuthology.orchestra.run.smithi170.stdout:(7/128): ceph-mon-19.0.0-3512.g5ccf1c30.el9.x86 11 MB/s | 4.7 MB 00:00 2024-05-08T04:05:58.795 INFO:teuthology.orchestra.run.smithi170.stdout:(8/128): ceph-common-19.0.0-3512.g5ccf1c30.el9. 9.5 MB/s | 21 MB 00:02 2024-05-08T04:05:58.937 INFO:teuthology.orchestra.run.smithi170.stdout:(9/128): ceph-selinux-19.0.0-3512.g5ccf1c30.el9 178 kB/s | 25 kB 00:00 2024-05-08T04:05:59.138 INFO:teuthology.orchestra.run.smithi170.stdout:(10/128): ceph-radosgw-19.0.0-3512.g5ccf1c30.el 8.8 MB/s | 11 MB 00:01 2024-05-08T04:05:59.338 INFO:teuthology.orchestra.run.smithi170.stdout:(11/128): ceph-osd-19.0.0-3512.g5ccf1c30.el9.x8 9.8 MB/s | 17 MB 00:01 2024-05-08T04:05:59.363 INFO:teuthology.orchestra.run.smithi170.stdout:(12/128): libcephfs-devel-19.0.0-3512.g5ccf1c30 141 kB/s | 32 kB 00:00 2024-05-08T04:05:59.597 INFO:teuthology.orchestra.run.smithi170.stdout:(13/128): libcephfs2-19.0.0-3512.g5ccf1c30.el9. 2.8 MB/s | 747 kB 00:00 2024-05-08T04:06:00.125 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - CRB 487 kB/s | 6.2 MB 00:13 2024-05-08T04:06:00.131 INFO:teuthology.orchestra.run.smithi170.stdout:(14/128): librados-devel-19.0.0-3512.g5ccf1c30. 238 kB/s | 127 kB 00:00 2024-05-08T04:06:00.156 INFO:teuthology.orchestra.run.smithi170.stdout:(15/128): libcephsqlite-19.0.0-3512.g5ccf1c30.e 206 kB/s | 164 kB 00:00 2024-05-08T04:06:00.239 INFO:teuthology.orchestra.run.smithi170.stdout:(16/128): libradosstriper1-19.0.0-3512.g5ccf1c3 4.5 MB/s | 500 kB 00:00 2024-05-08T04:06:00.556 INFO:teuthology.orchestra.run.smithi170.stdout:(17/128): python3-ceph-argparse-19.0.0-3512.g5c 142 kB/s | 45 kB 00:00 2024-05-08T04:06:00.740 INFO:teuthology.orchestra.run.smithi170.stdout:(18/128): python3-ceph-common-19.0.0-3512.g5ccf 740 kB/s | 135 kB 00:00 2024-05-08T04:06:00.824 INFO:teuthology.orchestra.run.smithi170.stdout:(19/128): librgw2-19.0.0-3512.g5ccf1c30.el9.x86 8.1 MB/s | 5.4 MB 00:00 2024-05-08T04:06:00.840 INFO:teuthology.orchestra.run.smithi170.stdout:(20/128): python3-cephfs-19.0.0-3512.g5ccf1c30. 1.6 MB/s | 161 kB 00:00 2024-05-08T04:06:00.891 INFO:teuthology.orchestra.run.smithi170.stdout:(21/128): python3-rados-19.0.0-3512.g5ccf1c30.e 4.7 MB/s | 321 kB 00:00 2024-05-08T04:06:00.933 INFO:teuthology.orchestra.run.smithi170.stdout:(22/128): python3-rbd-19.0.0-3512.g5ccf1c30.el9 3.2 MB/s | 297 kB 00:00 2024-05-08T04:06:00.991 INFO:teuthology.orchestra.run.smithi170.stdout:(23/128): python3-rgw-19.0.0-3512.g5ccf1c30.el9 992 kB/s | 100 kB 00:00 2024-05-08T04:06:01.017 INFO:teuthology.orchestra.run.smithi170.stdout:(24/128): rbd-fuse-19.0.0-3512.g5ccf1c30.el9.x8 1.0 MB/s | 85 kB 00:00 2024-05-08T04:06:01.242 INFO:teuthology.orchestra.run.smithi170.stdout:(25/128): rbd-nbd-19.0.0-3512.g5ccf1c30.el9.x86 757 kB/s | 170 kB 00:00 2024-05-08T04:06:01.317 INFO:teuthology.orchestra.run.smithi170.stdout:(26/128): rbd-mirror-19.0.0-3512.g5ccf1c30.el9. 9.5 MB/s | 3.1 MB 00:00 2024-05-08T04:06:01.343 INFO:teuthology.orchestra.run.smithi170.stdout:(27/128): ceph-grafana-dashboards-19.0.0-3512.g 341 kB/s | 34 kB 00:00 2024-05-08T04:06:01.409 INFO:teuthology.orchestra.run.smithi170.stdout:(28/128): ceph-mgr-cephadm-19.0.0-3512.g5ccf1c3 1.5 MB/s | 142 kB 00:00 2024-05-08T04:06:01.526 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS Stream 9 - Extras packages 75 kB/s | 16 kB 00:00 2024-05-08T04:06:01.651 INFO:teuthology.orchestra.run.smithi170.stdout:(29/128): ceph-mgr-dashboard-19.0.0-3512.g5ccf1 11 MB/s | 3.5 MB 00:00 2024-05-08T04:06:02.368 INFO:teuthology.orchestra.run.smithi170.stdout:(30/128): ceph-test-19.0.0-3512.g5ccf1c30.el9.x 14 MB/s | 49 MB 00:03 2024-05-08T04:06:02.394 INFO:teuthology.orchestra.run.smithi170.stdout:(31/128): ceph-mgr-modules-core-19.0.0-3512.g5c 369 kB/s | 274 kB 00:00 2024-05-08T04:06:02.436 INFO:teuthology.orchestra.run.smithi170.stdout:(32/128): ceph-mgr-rook-19.0.0-3512.g5ccf1c30.e 740 kB/s | 49 kB 00:00 2024-05-08T04:06:02.611 INFO:teuthology.orchestra.run.smithi170.stdout:(33/128): ceph-mgr-diskprediction-local-19.0.0- 6.2 MB/s | 7.4 MB 00:01 2024-05-08T04:06:02.636 INFO:teuthology.orchestra.run.smithi170.stdout:(34/128): ceph-prometheus-alerts-19.0.0-3512.g5 68 kB/s | 16 kB 00:00 2024-05-08T04:06:02.670 INFO:teuthology.orchestra.run.smithi170.stdout:(35/128): ceph-volume-19.0.0-3512.g5ccf1c30.el9 1.1 MB/s | 273 kB 00:00 2024-05-08T04:06:02.720 INFO:teuthology.orchestra.run.smithi170.stdout:(36/128): cephadm-19.0.0-3512.g5ccf1c30.el9.noa 6.9 MB/s | 764 kB 00:00 2024-05-08T04:06:03.371 INFO:teuthology.orchestra.run.smithi170.stdout:(37/128): fuse-2.9.9-15.el9.x86_64.rpm 109 kB/s | 80 kB 00:00 2024-05-08T04:06:03.563 INFO:teuthology.orchestra.run.smithi170.stdout:(38/128): libquadmath-11.4.1-3.el9.x86_64.rpm 232 kB/s | 195 kB 00:00 2024-05-08T04:06:03.654 INFO:teuthology.orchestra.run.smithi170.stdout:(39/128): libgfortran-11.4.1-3.el9.x86_64.rpm 816 kB/s | 803 kB 00:00 2024-05-08T04:06:04.204 INFO:teuthology.orchestra.run.smithi170.stdout:(40/128): python3-cffi-1.14.5-5.el9.x86_64.rpm 304 kB/s | 253 kB 00:00 2024-05-08T04:06:04.455 INFO:teuthology.orchestra.run.smithi170.stdout:(41/128): python3-ply-3.11-14.el9.noarch.rpm 133 kB/s | 106 kB 00:00 2024-05-08T04:06:04.672 INFO:teuthology.orchestra.run.smithi170.stdout:(42/128): python3-cryptography-36.0.1-4.el9.x86 1.1 MB/s | 1.2 MB 00:01 2024-05-08T04:06:05.064 INFO:teuthology.orchestra.run.smithi170.stdout:(43/128): python3-pycparser-2.20-6.el9.noarch.r 157 kB/s | 135 kB 00:00 2024-05-08T04:06:05.314 INFO:teuthology.orchestra.run.smithi170.stdout:(44/128): python3-requests-2.25.1-8.el9.noarch. 150 kB/s | 125 kB 00:00 2024-05-08T04:06:05.564 INFO:teuthology.orchestra.run.smithi170.stdout:(45/128): boost-program-options-1.75.0-8.el9.x8 214 kB/s | 107 kB 00:00 2024-05-08T04:06:05.589 INFO:teuthology.orchestra.run.smithi170.stdout:(46/128): flexiblas-3.0.4-8.el9.x86_64.rpm 119 kB/s | 33 kB 00:00 2024-05-08T04:06:05.615 INFO:teuthology.orchestra.run.smithi170.stdout:(47/128): python3-urllib3-1.26.5-5.el9.noarch.r 228 kB/s | 215 kB 00:00 2024-05-08T04:06:05.698 INFO:teuthology.orchestra.run.smithi170.stdout:(48/128): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-05-08T04:06:05.807 INFO:teuthology.orchestra.run.smithi170.stdout:(49/128): librabbitmq-0.11.0-7.el9.x86_64.rpm 418 kB/s | 45 kB 00:00 2024-05-08T04:06:06.132 INFO:teuthology.orchestra.run.smithi170.stdout:(50/128): libpmemobj-1.12.1-1.el9.x86_64.rpm 310 kB/s | 160 kB 00:00 2024-05-08T04:06:06.182 INFO:teuthology.orchestra.run.smithi170.stdout:(51/128): librdkafka-1.6.1-102.el9.x86_64.rpm 1.7 MB/s | 662 kB 00:00 2024-05-08T04:06:06.300 INFO:teuthology.orchestra.run.smithi170.stdout:(52/128): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.1 MB/s | 3.0 MB 00:00 2024-05-08T04:06:06.325 INFO:teuthology.orchestra.run.smithi170.stdout:(53/128): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.5 MB/s | 292 kB 00:00 2024-05-08T04:06:06.350 INFO:teuthology.orchestra.run.smithi170.stdout:(54/128): lua-5.4.4-4.el9.x86_64.rpm 1.1 MB/s | 188 kB 00:00 2024-05-08T04:06:06.400 INFO:teuthology.orchestra.run.smithi170.stdout:(55/128): openblas-0.3.21-2.el9.x86_64.rpm 351 kB/s | 35 kB 00:00 2024-05-08T04:06:06.550 INFO:teuthology.orchestra.run.smithi170.stdout:(56/128): protobuf-3.14.0-13.el9.x86_64.rpm 5.0 MB/s | 1.0 MB 00:00 2024-05-08T04:06:06.659 INFO:teuthology.orchestra.run.smithi170.stdout:(57/128): python3-devel-3.9.18-3.el9.x86_64.rpm 2.2 MB/s | 244 kB 00:00 2024-05-08T04:06:06.768 INFO:teuthology.orchestra.run.smithi170.stdout:(58/128): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-05-08T04:06:06.852 INFO:teuthology.orchestra.run.smithi187.stdout:Extra Packages for Enterprise Linux 4.1 MB/s | 21 MB 00:05 2024-05-08T04:06:06.868 INFO:teuthology.orchestra.run.smithi170.stdout:(59/128): python3-jsonpatch-1.21-16.el9.noarch. 255 kB/s | 26 kB 00:00 2024-05-08T04:06:06.968 INFO:teuthology.orchestra.run.smithi170.stdout:(60/128): python3-jsonpointer-2.0-4.el9.noarch. 192 kB/s | 19 kB 00:00 2024-05-08T04:06:07.077 INFO:teuthology.orchestra.run.smithi170.stdout:(61/128): python3-mako-1.1.4-6.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-05-08T04:06:07.177 INFO:teuthology.orchestra.run.smithi170.stdout:(62/128): python3-markupsafe-1.1.1-12.el9.x86_6 348 kB/s | 35 kB 00:00 2024-05-08T04:06:08.069 INFO:teuthology.orchestra.run.smithi170.stdout:(63/128): openblas-openmp-0.3.21-2.el9.x86_64.r 2.7 MB/s | 4.7 MB 00:01 2024-05-08T04:06:08.428 INFO:teuthology.orchestra.run.smithi170.stdout:(64/128): python3-numpy-f2py-1.20.1-5.el9.x86_6 700 kB/s | 250 kB 00:00 2024-05-08T04:06:08.770 INFO:teuthology.orchestra.run.smithi170.stdout:(65/128): python3-oauthlib-3.1.1-5.el9.noarch.r 650 kB/s | 222 kB 00:00 2024-05-08T04:06:08.953 INFO:teuthology.orchestra.run.smithi170.stdout:(66/128): python3-packaging-20.9-5.el9.noarch.r 421 kB/s | 77 kB 00:00 2024-05-08T04:06:09.070 INFO:teuthology.orchestra.run.smithi170.stdout:(67/128): python3-prettytable-0.7.2-27.el9.noar 362 kB/s | 42 kB 00:00 2024-05-08T04:06:09.496 INFO:teuthology.orchestra.run.smithi170.stdout:(68/128): python3-protobuf-3.14.0-13.el9.noarch 634 kB/s | 269 kB 00:00 2024-05-08T04:06:09.771 INFO:teuthology.orchestra.run.smithi170.stdout:(69/128): python3-pyasn1-0.4.8-6.el9.noarch.rpm 577 kB/s | 159 kB 00:00 2024-05-08T04:06:09.996 INFO:teuthology.orchestra.run.smithi170.stdout:(70/128): python3-numpy-1.20.1-5.el9.x86_64.rpm 1.8 MB/s | 5.1 MB 00:02 2024-05-08T04:06:10.088 INFO:teuthology.orchestra.run.smithi170.stdout:(71/128): python3-pytz-2021.1-5.el9.noarch.rpm 557 kB/s | 51 kB 00:00 2024-05-08T04:06:10.180 INFO:teuthology.orchestra.run.smithi170.stdout:(72/128): python3-requests-oauthlib-1.3.0-12.el 586 kB/s | 54 kB 00:00 2024-05-08T04:06:10.239 INFO:teuthology.orchestra.run.smithi170.stdout:(73/128): python3-pyasn1-modules-0.4.8-6.el9.no 596 kB/s | 279 kB 00:00 2024-05-08T04:06:10.339 INFO:teuthology.orchestra.run.smithi170.stdout:(74/128): python3-toml-0.10.2-6.el9.noarch.rpm 416 kB/s | 42 kB 00:00 2024-05-08T04:06:10.689 INFO:teuthology.orchestra.run.smithi170.stdout:(75/128): qatlib-24.02.0-1.el9.x86_64.rpm 631 kB/s | 221 kB 00:00 2024-05-08T04:06:10.789 INFO:teuthology.orchestra.run.smithi170.stdout:(76/128): qatlib-service-24.02.0-1.el9.x86_64.r 359 kB/s | 36 kB 00:00 2024-05-08T04:06:10.890 INFO:teuthology.orchestra.run.smithi170.stdout:(77/128): qatzip-libs-1.2.0-1.el9.x86_64.rpm 473 kB/s | 47 kB 00:00 2024-05-08T04:06:11.315 INFO:teuthology.orchestra.run.smithi170.stdout:(78/128): socat-1.7.4.1-5.el9.x86_64.rpm 717 kB/s | 305 kB 00:00 2024-05-08T04:06:11.974 INFO:teuthology.orchestra.run.smithi170.stdout:(79/128): python3-babel-2.9.1-2.el9.noarch.rpm 1.1 MB/s | 6.0 MB 00:05 2024-05-08T04:06:12.043 INFO:teuthology.orchestra.run.smithi187.stdout:lab-extras 45 kB/s | 1.7 kB 00:00 2024-05-08T04:06:12.257 INFO:teuthology.orchestra.run.smithi170.stdout:(80/128): sqlite-devel-3.34.1-7.el9.x86_64.rpm 474 kB/s | 134 kB 00:00 2024-05-08T04:06:12.382 INFO:teuthology.orchestra.run.smithi170.stdout:(81/128): sqlite-3.34.1-7.el9.x86_64.rpm 701 kB/s | 748 kB 00:01 2024-05-08T04:06:12.433 INFO:teuthology.orchestra.run.smithi170.stdout:(82/128): xmlstarlet-1.6.1-20.el9.x86_64.rpm 363 kB/s | 64 kB 00:00 2024-05-08T04:06:12.458 INFO:teuthology.orchestra.run.smithi170.stdout:(83/128): lua-devel-5.4.4-4.el9.x86_64.rpm 296 kB/s | 22 kB 00:00 2024-05-08T04:06:12.742 INFO:teuthology.orchestra.run.smithi170.stdout:(84/128): abseil-cpp-20211102.0-3.el9.x86_64.rp 1.9 MB/s | 548 kB 00:00 2024-05-08T04:06:12.775 INFO:teuthology.orchestra.run.smithi170.stdout:(85/128): protobuf-compiler-3.14.0-13.el9.x86_6 2.5 MB/s | 863 kB 00:00 2024-05-08T04:06:12.809 INFO:teuthology.orchestra.run.smithi170.stdout:(86/128): gperftools-libs-2.9.1-2.el9.x86_64.rp 4.5 MB/s | 309 kB 00:00 2024-05-08T04:06:12.901 INFO:teuthology.orchestra.run.smithi170.stdout:(87/128): grpc-data-1.46.7-10.el9.noarch.rpm 155 kB/s | 19 kB 00:00 2024-05-08T04:06:13.010 INFO:teuthology.orchestra.run.smithi170.stdout:(88/128): libarrow-9.0.0-11.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-05-08T04:06:13.043 INFO:teuthology.orchestra.run.smithi170.stdout:(89/128): libarrow-doc-9.0.0-11.el9.noarch.rpm 183 kB/s | 26 kB 00:00 2024-05-08T04:06:13.069 INFO:teuthology.orchestra.run.smithi170.stdout:(90/128): liboath-2.6.7-2.el9.x86_64.rpm 828 kB/s | 49 kB 00:00 2024-05-08T04:06:13.127 INFO:teuthology.orchestra.run.smithi170.stdout:(91/128): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-05-08T04:06:13.153 INFO:teuthology.orchestra.run.smithi170.stdout:(92/128): libunwind-1.6.2-1.el9.x86_64.rpm 619 kB/s | 67 kB 00:00 2024-05-08T04:06:13.211 INFO:teuthology.orchestra.run.smithi170.stdout:(93/128): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 839 kB 00:00 2024-05-08T04:06:13.261 INFO:teuthology.orchestra.run.smithi170.stdout:(94/128): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 871 kB/s | 43 kB 00:00 2024-05-08T04:06:13.304 INFO:teuthology.orchestra.run.smithi170.stdout:(95/128): python3-asyncssh-2.13.2-5.el9.noarch. 3.6 MB/s | 548 kB 00:00 2024-05-08T04:06:13.329 INFO:teuthology.orchestra.run.smithi170.stdout:(96/128): python3-cachetools-4.2.4-1.el9.noarch 481 kB/s | 32 kB 00:00 2024-05-08T04:06:13.354 INFO:teuthology.orchestra.run.smithi170.stdout:(97/128): python3-certifi-2021.10.8-2.el9.noarc 299 kB/s | 15 kB 00:00 2024-05-08T04:06:13.387 INFO:teuthology.orchestra.run.smithi170.stdout:(98/128): python3-cheroot-8.6.0-4.el9.noarch.rp 2.9 MB/s | 172 kB 00:00 2024-05-08T04:06:13.430 INFO:teuthology.orchestra.run.smithi170.stdout:(99/128): python3-cherrypy-18.6.1-2.el9.noarch. 4.7 MB/s | 358 kB 00:00 2024-05-08T04:06:13.455 INFO:teuthology.orchestra.run.smithi170.stdout:(100/128): python3-google-auth-2.29.0-1.el9.noa 3.0 MB/s | 206 kB 00:00 2024-05-08T04:06:13.514 INFO:teuthology.orchestra.run.smithi170.stdout:(101/128): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-05-08T04:06:13.572 INFO:teuthology.orchestra.run.smithi170.stdout:(102/128): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-05-08T04:06:13.589 INFO:teuthology.orchestra.run.smithi170.stdout:(103/128): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-05-08T04:06:13.614 INFO:teuthology.orchestra.run.smithi170.stdout:(104/128): python3-jaraco-classes-3.2.1-5.el9.n 422 kB/s | 18 kB 00:00 2024-05-08T04:06:13.631 INFO:teuthology.orchestra.run.smithi170.stdout:(105/128): python3-jaraco-collections-3.0.0-8.e 553 kB/s | 23 kB 00:00 2024-05-08T04:06:13.657 INFO:teuthology.orchestra.run.smithi170.stdout:(106/128): python3-jaraco-functools-3.5.0-2.el9 464 kB/s | 19 kB 00:00 2024-05-08T04:06:13.673 INFO:teuthology.orchestra.run.smithi170.stdout:(107/128): python3-jaraco-text-3.2.0-6.el9.noar 470 kB/s | 20 kB 00:00 2024-05-08T04:06:13.732 INFO:teuthology.orchestra.run.smithi170.stdout:(108/128): python3-kubernetes-26.1.0-2.el9.noar 14 MB/s | 1.0 MB 00:00 2024-05-08T04:06:13.749 INFO:teuthology.orchestra.run.smithi170.stdout:(109/128): python3-logutils-0.3.5-21.el9.noarch 615 kB/s | 46 kB 00:00 2024-05-08T04:06:13.783 INFO:teuthology.orchestra.run.smithi170.stdout:(110/128): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-05-08T04:06:13.800 INFO:teuthology.orchestra.run.smithi170.stdout:(111/128): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-05-08T04:06:13.842 INFO:teuthology.orchestra.run.smithi170.stdout:(112/128): python3-pecan-1.4.2-3.el9.noarch.rpm 4.6 MB/s | 272 kB 00:00 2024-05-08T04:06:13.875 INFO:teuthology.orchestra.run.smithi170.stdout:(113/128): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-05-08T04:06:13.901 INFO:teuthology.orchestra.run.smithi170.stdout:(114/128): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-05-08T04:06:13.926 INFO:teuthology.orchestra.run.smithi170.stdout:(115/128): python3-repoze-lru-0.7-16.el9.noarch 608 kB/s | 31 kB 00:00 2024-05-08T04:06:13.960 INFO:teuthology.orchestra.run.smithi170.stdout:(116/128): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-05-08T04:06:13.977 INFO:teuthology.orchestra.run.smithi170.stdout:(117/128): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-05-08T04:06:14.002 INFO:teuthology.orchestra.run.smithi170.stdout:(118/128): python3-tempora-5.0.0-2.el9.noarch.r 857 kB/s | 36 kB 00:00 2024-05-08T04:06:14.019 INFO:teuthology.orchestra.run.smithi170.stdout:(119/128): python3-typing-extensions-4.4.0-2.el 1.2 MB/s | 51 kB 00:00 2024-05-08T04:06:14.053 INFO:teuthology.orchestra.run.smithi170.stdout:(120/128): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-05-08T04:06:14.078 INFO:teuthology.orchestra.run.smithi170.stdout:(121/128): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-05-08T04:06:14.111 INFO:teuthology.orchestra.run.smithi170.stdout:(122/128): python3-werkzeug-2.0.3-3.el9.1.noarc 7.1 MB/s | 427 kB 00:00 2024-05-08T04:06:14.128 INFO:teuthology.orchestra.run.smithi170.stdout:(123/128): python3-zc-lockfile-2.0-10.el9.noarc 396 kB/s | 20 kB 00:00 2024-05-08T04:06:14.170 INFO:teuthology.orchestra.run.smithi170.stdout:(124/128): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-05-08T04:06:14.212 INFO:teuthology.orchestra.run.smithi170.stdout:(125/128): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-05-08T04:06:14.353 INFO:teuthology.orchestra.run.smithi187.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T04:06:14.354 INFO:teuthology.orchestra.run.smithi187.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T04:06:14.413 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2024-05-08T04:06:14.431 INFO:teuthology.orchestra.run.smithi187.stdout:======================================================================================= 2024-05-08T04:06:14.431 INFO:teuthology.orchestra.run.smithi187.stdout: Package Arch Version Repository Size 2024-05-08T04:06:14.431 INFO:teuthology.orchestra.run.smithi187.stdout:======================================================================================= 2024-05-08T04:06:14.431 INFO:teuthology.orchestra.run.smithi187.stdout:Installing: 2024-05-08T04:06:14.431 INFO:teuthology.orchestra.run.smithi187.stdout: ceph x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 6.6 k 2024-05-08T04:06:14.431 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-base x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 5.4 M 2024-05-08T04:06:14.431 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-fuse x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 890 k 2024-05-08T04:06:14.431 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 145 k 2024-05-08T04:06:14.431 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 1.5 M 2024-05-08T04:06:14.431 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3512.g5ccf1c30.el9 ceph-noarch 142 k 2024-05-08T04:06:14.432 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3512.g5ccf1c30.el9 ceph-noarch 3.5 M 2024-05-08T04:06:14.432 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3512.g5ccf1c30.el9 ceph-noarch 7.4 M 2024-05-08T04:06:14.432 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-rook noarch 2:19.0.0-3512.g5ccf1c30.el9 ceph-noarch 49 k 2024-05-08T04:06:14.432 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-radosgw x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 11 M 2024-05-08T04:06:14.432 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-test x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 49 M 2024-05-08T04:06:14.432 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-volume noarch 2:19.0.0-3512.g5ccf1c30.el9 ceph-noarch 273 k 2024-05-08T04:06:14.432 INFO:teuthology.orchestra.run.smithi187.stdout: cephadm noarch 2:19.0.0-3512.g5ccf1c30.el9 ceph-noarch 764 k 2024-05-08T04:06:14.432 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs-devel x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 32 k 2024-05-08T04:06:14.432 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs2 x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 747 k 2024-05-08T04:06:14.432 INFO:teuthology.orchestra.run.smithi187.stdout: librados-devel x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 127 k 2024-05-08T04:06:14.432 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cephfs x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 161 k 2024-05-08T04:06:14.432 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rados x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 321 k 2024-05-08T04:06:14.432 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rbd x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 297 k 2024-05-08T04:06:14.432 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rgw x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 100 k 2024-05-08T04:06:14.432 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-fuse x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 85 k 2024-05-08T04:06:14.432 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-mirror x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 3.1 M 2024-05-08T04:06:14.433 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-nbd x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 170 k 2024-05-08T04:06:14.433 INFO:teuthology.orchestra.run.smithi187.stdout: sqlite-devel x86_64 3.34.1-7.el9 appstream 134 k 2024-05-08T04:06:14.433 INFO:teuthology.orchestra.run.smithi187.stdout:Upgrading: 2024-05-08T04:06:14.433 INFO:teuthology.orchestra.run.smithi187.stdout: librados2 x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 3.4 M 2024-05-08T04:06:14.433 INFO:teuthology.orchestra.run.smithi187.stdout: librbd1 x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 3.1 M 2024-05-08T04:06:14.433 INFO:teuthology.orchestra.run.smithi187.stdout:Installing dependencies: 2024-05-08T04:06:14.433 INFO:teuthology.orchestra.run.smithi187.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T04:06:14.433 INFO:teuthology.orchestra.run.smithi187.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T04:06:14.433 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-common x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 21 M 2024-05-08T04:06:14.433 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3512.g5ccf1c30.el9 ceph-noarch 34 k 2024-05-08T04:06:14.433 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mds x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 2.4 M 2024-05-08T04:06:14.433 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3512.g5ccf1c30.el9 ceph-noarch 274 k 2024-05-08T04:06:14.433 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mon x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 4.7 M 2024-05-08T04:06:14.434 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-osd x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 17 M 2024-05-08T04:06:14.434 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3512.g5ccf1c30.el9 ceph-noarch 16 k 2024-05-08T04:06:14.434 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-selinux x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 25 k 2024-05-08T04:06:14.434 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T04:06:14.434 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T04:06:14.434 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T04:06:14.434 INFO:teuthology.orchestra.run.smithi187.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T04:06:14.434 INFO:teuthology.orchestra.run.smithi187.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T04:06:14.434 INFO:teuthology.orchestra.run.smithi187.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T04:06:14.434 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T04:06:14.434 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T04:06:14.434 INFO:teuthology.orchestra.run.smithi187.stdout: libcephsqlite x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 164 k 2024-05-08T04:06:14.434 INFO:teuthology.orchestra.run.smithi187.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T04:06:14.435 INFO:teuthology.orchestra.run.smithi187.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T04:06:14.435 INFO:teuthology.orchestra.run.smithi187.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T04:06:14.435 INFO:teuthology.orchestra.run.smithi187.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T04:06:14.435 INFO:teuthology.orchestra.run.smithi187.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T04:06:14.435 INFO:teuthology.orchestra.run.smithi187.stdout: libradosstriper1 x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 500 k 2024-05-08T04:06:14.435 INFO:teuthology.orchestra.run.smithi187.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T04:06:14.435 INFO:teuthology.orchestra.run.smithi187.stdout: librgw2 x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 5.4 M 2024-05-08T04:06:14.435 INFO:teuthology.orchestra.run.smithi187.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T04:06:14.435 INFO:teuthology.orchestra.run.smithi187.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T04:06:14.435 INFO:teuthology.orchestra.run.smithi187.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T04:06:14.435 INFO:teuthology.orchestra.run.smithi187.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T04:06:14.435 INFO:teuthology.orchestra.run.smithi187.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T04:06:14.435 INFO:teuthology.orchestra.run.smithi187.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T04:06:14.435 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T04:06:14.436 INFO:teuthology.orchestra.run.smithi187.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T04:06:14.436 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T04:06:14.436 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T04:06:14.436 INFO:teuthology.orchestra.run.smithi187.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T04:06:14.436 INFO:teuthology.orchestra.run.smithi187.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T04:06:14.436 INFO:teuthology.orchestra.run.smithi187.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T04:06:14.436 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T04:06:14.436 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-argparse x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 45 k 2024-05-08T04:06:14.436 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-common x86_64 2:19.0.0-3512.g5ccf1c30.el9 ceph 135 k 2024-05-08T04:06:14.436 INFO:teuthology.orchestra.run.smithi187.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T04:06:14.436 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T04:06:14.436 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T04:06:14.436 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T04:06:14.436 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T04:06:14.437 INFO:teuthology.orchestra.run.smithi187.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T04:06:14.437 INFO:teuthology.orchestra.run.smithi187.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T04:06:14.437 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T04:06:14.437 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T04:06:14.437 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T04:06:14.437 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T04:06:14.437 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-08T04:06:14.437 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T04:06:14.437 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T04:06:14.437 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T04:06:14.437 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T04:06:14.437 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T04:06:14.437 INFO:teuthology.orchestra.run.smithi187.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T04:06:14.437 INFO:teuthology.orchestra.run.smithi187.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T04:06:14.438 INFO:teuthology.orchestra.run.smithi187.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T04:06:14.438 INFO:teuthology.orchestra.run.smithi187.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T04:06:14.438 INFO:teuthology.orchestra.run.smithi187.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T04:06:14.438 INFO:teuthology.orchestra.run.smithi187.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T04:06:14.438 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T04:06:14.438 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T04:06:14.438 INFO:teuthology.orchestra.run.smithi187.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T04:06:14.438 INFO:teuthology.orchestra.run.smithi187.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T04:06:14.438 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T04:06:14.438 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T04:06:14.438 INFO:teuthology.orchestra.run.smithi187.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T04:06:14.438 INFO:teuthology.orchestra.run.smithi187.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T04:06:14.438 INFO:teuthology.orchestra.run.smithi187.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T04:06:14.438 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T04:06:14.439 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T04:06:14.439 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T04:06:14.439 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T04:06:14.439 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T04:06:14.439 INFO:teuthology.orchestra.run.smithi187.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T04:06:14.439 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T04:06:14.439 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T04:06:14.439 INFO:teuthology.orchestra.run.smithi187.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T04:06:14.439 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T04:06:14.439 INFO:teuthology.orchestra.run.smithi187.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T04:06:14.439 INFO:teuthology.orchestra.run.smithi187.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T04:06:14.439 INFO:teuthology.orchestra.run.smithi187.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T04:06:14.439 INFO:teuthology.orchestra.run.smithi187.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T04:06:14.439 INFO:teuthology.orchestra.run.smithi187.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T04:06:14.439 INFO:teuthology.orchestra.run.smithi187.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T04:06:14.440 INFO:teuthology.orchestra.run.smithi187.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T04:06:14.440 INFO:teuthology.orchestra.run.smithi187.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T04:06:14.440 INFO:teuthology.orchestra.run.smithi187.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T04:06:14.440 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T04:06:14.440 INFO:teuthology.orchestra.run.smithi187.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T04:06:14.440 INFO:teuthology.orchestra.run.smithi187.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T04:06:14.440 INFO:teuthology.orchestra.run.smithi187.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T04:06:14.440 INFO:teuthology.orchestra.run.smithi187.stdout: sqlite x86_64 3.34.1-7.el9 appstream 748 k 2024-05-08T04:06:14.440 INFO:teuthology.orchestra.run.smithi187.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T04:06:14.440 INFO:teuthology.orchestra.run.smithi187.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T04:06:14.440 INFO:teuthology.orchestra.run.smithi187.stdout:Installing weak dependencies: 2024-05-08T04:06:14.440 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T04:06:14.440 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:06:14.440 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2024-05-08T04:06:14.440 INFO:teuthology.orchestra.run.smithi187.stdout:======================================================================================= 2024-05-08T04:06:14.441 INFO:teuthology.orchestra.run.smithi187.stdout:Install 126 Packages 2024-05-08T04:06:14.441 INFO:teuthology.orchestra.run.smithi187.stdout:Upgrade 2 Packages 2024-05-08T04:06:14.441 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:06:14.441 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 203 M 2024-05-08T04:06:14.441 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2024-05-08T04:06:14.463 INFO:teuthology.orchestra.run.smithi170.stdout:(126/128): librados2-19.0.0-3512.g5ccf1c30.el9. 12 MB/s | 3.4 MB 00:00 2024-05-08T04:06:14.688 INFO:teuthology.orchestra.run.smithi170.stdout:(127/128): librbd1-19.0.0-3512.g5ccf1c30.el9.x8 6.6 MB/s | 3.1 MB 00:00 2024-05-08T04:06:15.795 INFO:teuthology.orchestra.run.smithi187.stdout:(1/128): ceph-19.0.0-3512.g5ccf1c30.el9.x86_64. 14 kB/s | 6.6 kB 00:00 2024-05-08T04:06:16.196 INFO:teuthology.orchestra.run.smithi187.stdout:(2/128): ceph-base-19.0.0-3512.g5ccf1c30.el9.x8 6.1 MB/s | 5.4 MB 00:00 2024-05-08T04:06:16.229 INFO:teuthology.orchestra.run.smithi187.stdout:(3/128): ceph-fuse-19.0.0-3512.g5ccf1c30.el9.x8 2.0 MB/s | 890 kB 00:00 2024-05-08T04:06:16.254 INFO:teuthology.orchestra.run.smithi187.stdout:(4/128): ceph-immutable-object-cache-19.0.0-351 2.4 MB/s | 145 kB 00:00 2024-05-08T04:06:16.388 INFO:teuthology.orchestra.run.smithi187.stdout:(5/128): ceph-mgr-19.0.0-3512.g5ccf1c30.el9.x86 11 MB/s | 1.5 MB 00:00 2024-05-08T04:06:16.680 INFO:teuthology.orchestra.run.smithi187.stdout:(6/128): ceph-mon-19.0.0-3512.g5ccf1c30.el9.x86 16 MB/s | 4.7 MB 00:00 2024-05-08T04:06:16.730 INFO:teuthology.orchestra.run.smithi187.stdout:(7/128): ceph-mds-19.0.0-3512.g5ccf1c30.el9.x86 4.7 MB/s | 2.4 MB 00:00 2024-05-08T04:06:17.605 INFO:teuthology.orchestra.run.smithi187.stdout:(8/128): ceph-osd-19.0.0-3512.g5ccf1c30.el9.x86 18 MB/s | 17 MB 00:00 2024-05-08T04:06:17.655 INFO:teuthology.orchestra.run.smithi187.stdout:(9/128): ceph-selinux-19.0.0-3512.g5ccf1c30.el9 507 kB/s | 25 kB 00:00 2024-05-08T04:06:19.072 INFO:teuthology.orchestra.run.smithi187.stdout:(10/128): ceph-radosgw-19.0.0-3512.g5ccf1c30.el 4.5 MB/s | 11 MB 00:02 2024-05-08T04:06:19.114 INFO:teuthology.orchestra.run.smithi187.stdout:(11/128): libcephfs-devel-19.0.0-3512.g5ccf1c30 766 kB/s | 32 kB 00:00 2024-05-08T04:06:19.314 INFO:teuthology.orchestra.run.smithi187.stdout:(12/128): libcephfs2-19.0.0-3512.g5ccf1c30.el9. 3.7 MB/s | 747 kB 00:00 2024-05-08T04:06:19.390 INFO:teuthology.orchestra.run.smithi187.stdout:(13/128): libcephsqlite-19.0.0-3512.g5ccf1c30.e 2.1 MB/s | 164 kB 00:00 2024-05-08T04:06:19.473 INFO:teuthology.orchestra.run.smithi187.stdout:(14/128): librados-devel-19.0.0-3512.g5ccf1c30. 1.5 MB/s | 127 kB 00:00 2024-05-08T04:06:19.632 INFO:teuthology.orchestra.run.smithi187.stdout:(15/128): libradosstriper1-19.0.0-3512.g5ccf1c3 3.1 MB/s | 500 kB 00:00 2024-05-08T04:06:20.516 INFO:teuthology.orchestra.run.smithi187.stdout:(16/128): ceph-test-19.0.0-3512.g5ccf1c30.el9.x 17 MB/s | 49 MB 00:02 2024-05-08T04:06:20.566 INFO:teuthology.orchestra.run.smithi187.stdout:(17/128): python3-ceph-argparse-19.0.0-3512.g5c 912 kB/s | 45 kB 00:00 2024-05-08T04:06:20.658 INFO:teuthology.orchestra.run.smithi187.stdout:(18/128): librgw2-19.0.0-3512.g5ccf1c30.el9.x86 5.2 MB/s | 5.4 MB 00:01 2024-05-08T04:06:20.691 INFO:teuthology.orchestra.run.smithi187.stdout:(19/128): python3-ceph-common-19.0.0-3512.g5ccf 1.1 MB/s | 135 kB 00:00 2024-05-08T04:06:20.717 INFO:teuthology.orchestra.run.smithi187.stdout:(20/128): python3-cephfs-19.0.0-3512.g5ccf1c30. 2.7 MB/s | 161 kB 00:00 2024-05-08T04:06:20.742 INFO:teuthology.orchestra.run.smithi187.stdout:(21/128): python3-rados-19.0.0-3512.g5ccf1c30.e 6.3 MB/s | 321 kB 00:00 2024-05-08T04:06:20.776 INFO:teuthology.orchestra.run.smithi187.stdout:(22/128): python3-rbd-19.0.0-3512.g5ccf1c30.el9 4.9 MB/s | 297 kB 00:00 2024-05-08T04:06:20.801 INFO:teuthology.orchestra.run.smithi187.stdout:(23/128): python3-rgw-19.0.0-3512.g5ccf1c30.el9 1.7 MB/s | 100 kB 00:00 2024-05-08T04:06:20.826 INFO:teuthology.orchestra.run.smithi187.stdout:(24/128): rbd-fuse-19.0.0-3512.g5ccf1c30.el9.x8 1.7 MB/s | 85 kB 00:00 2024-05-08T04:06:20.902 INFO:teuthology.orchestra.run.smithi187.stdout:(25/128): rbd-nbd-19.0.0-3512.g5ccf1c30.el9.x86 2.2 MB/s | 170 kB 00:00 2024-05-08T04:06:20.993 INFO:teuthology.orchestra.run.smithi187.stdout:(26/128): rbd-mirror-19.0.0-3512.g5ccf1c30.el9. 16 MB/s | 3.1 MB 00:00 2024-05-08T04:06:21.019 INFO:teuthology.orchestra.run.smithi187.stdout:(27/128): ceph-grafana-dashboards-19.0.0-3512.g 293 kB/s | 34 kB 00:00 2024-05-08T04:06:21.044 INFO:teuthology.orchestra.run.smithi187.stdout:(28/128): ceph-mgr-cephadm-19.0.0-3512.g5ccf1c3 2.8 MB/s | 142 kB 00:00 2024-05-08T04:06:21.494 INFO:teuthology.orchestra.run.smithi187.stdout:(29/128): ceph-mgr-diskprediction-local-19.0.0- 16 MB/s | 7.4 MB 00:00 2024-05-08T04:06:21.553 INFO:teuthology.orchestra.run.smithi187.stdout:(30/128): ceph-mgr-modules-core-19.0.0-3512.g5c 4.6 MB/s | 274 kB 00:00 2024-05-08T04:06:21.612 INFO:teuthology.orchestra.run.smithi187.stdout:(31/128): ceph-mgr-dashboard-19.0.0-3512.g5ccf1 6.0 MB/s | 3.5 MB 00:00 2024-05-08T04:06:21.637 INFO:teuthology.orchestra.run.smithi187.stdout:(32/128): ceph-mgr-rook-19.0.0-3512.g5ccf1c30.e 592 kB/s | 49 kB 00:00 2024-05-08T04:06:21.662 INFO:teuthology.orchestra.run.smithi187.stdout:(33/128): ceph-prometheus-alerts-19.0.0-3512.g5 329 kB/s | 16 kB 00:00 2024-05-08T04:06:21.687 INFO:teuthology.orchestra.run.smithi187.stdout:(34/128): ceph-volume-19.0.0-3512.g5ccf1c30.el9 5.3 MB/s | 273 kB 00:00 2024-05-08T04:06:21.754 INFO:teuthology.orchestra.run.smithi187.stdout:(35/128): cephadm-19.0.0-3512.g5ccf1c30.el9.noa 8.1 MB/s | 764 kB 00:00 2024-05-08T04:06:22.213 INFO:teuthology.orchestra.run.smithi187.stdout:(36/128): ceph-common-19.0.0-3512.g5ccf1c30.el9 3.1 MB/s | 21 MB 00:06 2024-05-08T04:06:22.238 INFO:teuthology.orchestra.run.smithi187.stdout:(37/128): fuse-2.9.9-15.el9.x86_64.rpm 146 kB/s | 80 kB 00:00 2024-05-08T04:06:22.597 INFO:teuthology.orchestra.run.smithi187.stdout:(38/128): python3-cffi-1.14.5-5.el9.x86_64.rpm 707 kB/s | 253 kB 00:00 2024-05-08T04:06:22.664 INFO:teuthology.orchestra.run.smithi187.stdout:(39/128): libgfortran-11.4.1-3.el9.x86_64.rpm 883 kB/s | 803 kB 00:00 2024-05-08T04:06:22.772 INFO:teuthology.orchestra.run.smithi187.stdout:(40/128): python3-ply-3.11-14.el9.noarch.rpm 984 kB/s | 106 kB 00:00 2024-05-08T04:06:22.797 INFO:teuthology.orchestra.run.smithi187.stdout:(41/128): libquadmath-11.4.1-3.el9.x86_64.rpm 334 kB/s | 195 kB 00:00 2024-05-08T04:06:22.881 INFO:teuthology.orchestra.run.smithi187.stdout:(42/128): python3-cryptography-36.0.1-4.el9.x86 4.3 MB/s | 1.2 MB 00:00 2024-05-08T04:06:22.906 INFO:teuthology.orchestra.run.smithi187.stdout:(43/128): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-05-08T04:06:22.932 INFO:teuthology.orchestra.run.smithi187.stdout:(44/128): python3-pycparser-2.20-6.el9.noarch.r 848 kB/s | 135 kB 00:00 2024-05-08T04:06:22.990 INFO:teuthology.orchestra.run.smithi187.stdout:(45/128): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-05-08T04:06:23.591 INFO:teuthology.orchestra.run.smithi187.stdout:(46/128): flexiblas-3.0.4-8.el9.x86_64.rpm 50 kB/s | 33 kB 00:00 2024-05-08T04:06:23.658 INFO:teuthology.orchestra.run.smithi187.stdout:(47/128): boost-program-options-1.75.0-8.el9.x8 142 kB/s | 107 kB 00:00 2024-05-08T04:06:24.099 INFO:teuthology.orchestra.run.smithi187.stdout:(48/128): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.7 MB/s | 3.0 MB 00:01 2024-05-08T04:06:24.125 INFO:teuthology.orchestra.run.smithi187.stdout:(49/128): flexiblas-openblas-openmp-3.0.4-8.el9 34 kB/s | 18 kB 00:00 2024-05-08T04:06:24.458 INFO:teuthology.orchestra.run.smithi187.stdout:(50/128): libpmemobj-1.12.1-1.el9.x86_64.rpm 200 kB/s | 160 kB 00:00 2024-05-08T04:06:24.733 INFO:teuthology.orchestra.run.smithi187.stdout:(51/128): librabbitmq-0.11.0-7.el9.x86_64.rpm 71 kB/s | 45 kB 00:00 2024-05-08T04:06:25.067 INFO:teuthology.orchestra.run.smithi187.stdout:(52/128): librdkafka-1.6.1-102.el9.x86_64.rpm 703 kB/s | 662 kB 00:00 2024-05-08T04:06:25.276 INFO:teuthology.orchestra.run.smithi187.stdout:(53/128): lttng-ust-2.12.0-6.el9.x86_64.rpm 358 kB/s | 292 kB 00:00 2024-05-08T04:06:25.551 INFO:teuthology.orchestra.run.smithi187.stdout:(54/128): lua-5.4.4-4.el9.x86_64.rpm 230 kB/s | 188 kB 00:00 2024-05-08T04:06:25.676 INFO:teuthology.orchestra.run.smithi187.stdout:(55/128): openblas-0.3.21-2.el9.x86_64.rpm 58 kB/s | 35 kB 00:00 2024-05-08T04:06:26.426 INFO:teuthology.orchestra.run.smithi187.stdout:(56/128): openblas-openmp-0.3.21-2.el9.x86_64.r 4.1 MB/s | 4.7 MB 00:01 2024-05-08T04:06:26.543 INFO:teuthology.orchestra.run.smithi187.stdout:(57/128): protobuf-3.14.0-13.el9.x86_64.rpm 1.0 MB/s | 1.0 MB 00:00 2024-05-08T04:06:26.885 INFO:teuthology.orchestra.run.smithi187.stdout:(58/128): python3-babel-2.9.1-2.el9.noarch.rpm 4.9 MB/s | 6.0 MB 00:01 2024-05-08T04:06:27.285 INFO:teuthology.orchestra.run.smithi187.stdout:(59/128): python3-devel-3.9.18-3.el9.x86_64.rpm 284 kB/s | 244 kB 00:00 2024-05-08T04:06:27.361 INFO:teuthology.orchestra.run.smithi187.stdout:(60/128): python3-jinja2-2.11.3-5.el9.noarch.rp 303 kB/s | 248 kB 00:00 2024-05-08T04:06:27.486 INFO:teuthology.orchestra.run.smithi187.stdout:(61/128): python3-jsonpatch-1.21-16.el9.noarch. 43 kB/s | 26 kB 00:00 2024-05-08T04:06:27.820 INFO:teuthology.orchestra.run.smithi187.stdout:(62/128): python3-jsonpointer-2.0-4.el9.noarch. 36 kB/s | 19 kB 00:00 2024-05-08T04:06:28.095 INFO:teuthology.orchestra.run.smithi187.stdout:(63/128): python3-markupsafe-1.1.1-12.el9.x86_6 57 kB/s | 35 kB 00:00 2024-05-08T04:06:28.120 INFO:teuthology.orchestra.run.smithi187.stdout:(64/128): python3-mako-1.1.4-6.el9.noarch.rpm 227 kB/s | 172 kB 00:00 2024-05-08T04:06:28.895 INFO:teuthology.orchestra.run.smithi187.stdout:(65/128): python3-numpy-f2py-1.20.1-5.el9.x86_6 313 kB/s | 250 kB 00:00 2024-05-08T04:06:28.921 INFO:teuthology.orchestra.run.smithi187.stdout:(66/128): python3-oauthlib-3.1.1-5.el9.noarch.r 278 kB/s | 222 kB 00:00 2024-05-08T04:06:29.071 INFO:teuthology.orchestra.run.smithi187.stdout:(67/128): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.1 MB/s | 5.1 MB 00:01 2024-05-08T04:06:29.546 INFO:teuthology.orchestra.run.smithi187.stdout:(68/128): python3-prettytable-0.7.2-27.el9.noar 68 kB/s | 42 kB 00:00 2024-05-08T04:06:29.646 INFO:teuthology.orchestra.run.smithi187.stdout:(69/128): python3-packaging-20.9-5.el9.noarch.r 103 kB/s | 77 kB 00:00 2024-05-08T04:06:29.922 INFO:teuthology.orchestra.run.smithi187.stdout:(70/128): python3-protobuf-3.14.0-13.el9.noarch 317 kB/s | 269 kB 00:00 2024-05-08T04:06:30.380 INFO:teuthology.orchestra.run.smithi187.stdout:(71/128): python3-pyasn1-0.4.8-6.el9.noarch.rpm 190 kB/s | 159 kB 00:00 2024-05-08T04:06:30.463 INFO:teuthology.orchestra.run.smithi187.stdout:(72/128): python3-pyasn1-modules-0.4.8-6.el9.no 341 kB/s | 279 kB 00:00 2024-05-08T04:06:30.564 INFO:teuthology.orchestra.run.smithi187.stdout:(73/128): python3-pytz-2021.1-5.el9.noarch.rpm 79 kB/s | 51 kB 00:00 2024-05-08T04:06:31.064 INFO:teuthology.orchestra.run.smithi187.stdout:(74/128): python3-requests-oauthlib-1.3.0-12.el 78 kB/s | 54 kB 00:00 2024-05-08T04:06:31.173 INFO:teuthology.orchestra.run.smithi187.stdout:(75/128): python3-toml-0.10.2-6.el9.noarch.rpm 68 kB/s | 42 kB 00:00 2024-05-08T04:06:31.773 INFO:teuthology.orchestra.run.smithi187.stdout:(76/128): qatlib-service-24.02.0-1.el9.x86_64.r 60 kB/s | 36 kB 00:00 2024-05-08T04:06:31.857 INFO:teuthology.orchestra.run.smithi187.stdout:(77/128): qatlib-24.02.0-1.el9.x86_64.rpm 279 kB/s | 221 kB 00:00 2024-05-08T04:06:32.032 INFO:teuthology.orchestra.run.smithi187.stdout:(78/128): python3-scipy-1.6.2-8.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-05-08T04:06:32.086 INFO:teuthology.orchestra.run.smithi170.stdout:(128/128): python3-scipy-1.6.2-8.el9.x86_64.rpm 760 kB/s | 16 MB 00:21 2024-05-08T04:06:32.090 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2024-05-08T04:06:32.090 INFO:teuthology.orchestra.run.smithi170.stdout:Total 5.6 MB/s | 203 MB 00:36 2024-05-08T04:06:32.532 INFO:teuthology.orchestra.run.smithi187.stdout:(79/128): qatzip-libs-1.2.0-1.el9.x86_64.rpm 62 kB/s | 47 kB 00:00 2024-05-08T04:06:32.815 INFO:teuthology.orchestra.run.smithi187.stdout:(80/128): socat-1.7.4.1-5.el9.x86_64.rpm 318 kB/s | 305 kB 00:00 2024-05-08T04:06:32.872 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2024-05-08T04:06:32.973 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2024-05-08T04:06:32.973 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2024-05-08T04:06:33.024 INFO:teuthology.orchestra.run.smithi187.stdout:(81/128): sqlite-3.34.1-7.el9.x86_64.rpm 755 kB/s | 748 kB 00:00 2024-05-08T04:06:33.124 INFO:teuthology.orchestra.run.smithi187.stdout:(82/128): lua-devel-5.4.4-4.el9.x86_64.rpm 223 kB/s | 22 kB 00:00 2024-05-08T04:06:33.283 INFO:teuthology.orchestra.run.smithi187.stdout:(83/128): protobuf-compiler-3.14.0-13.el9.x86_6 5.3 MB/s | 863 kB 00:00 2024-05-08T04:06:33.341 INFO:teuthology.orchestra.run.smithi187.stdout:(84/128): sqlite-devel-3.34.1-7.el9.x86_64.rpm 166 kB/s | 134 kB 00:00 2024-05-08T04:06:33.592 INFO:teuthology.orchestra.run.smithi187.stdout:(85/128): xmlstarlet-1.6.1-20.el9.x86_64.rpm 82 kB/s | 64 kB 00:00 2024-05-08T04:06:33.833 INFO:teuthology.orchestra.run.smithi187.stdout:(86/128): grpc-data-1.46.7-10.el9.noarch.rpm 80 kB/s | 19 kB 00:00 2024-05-08T04:06:33.901 INFO:teuthology.orchestra.run.smithi187.stdout:(87/128): gperftools-libs-2.9.1-2.el9.x86_64.rp 553 kB/s | 309 kB 00:00 2024-05-08T04:06:33.926 INFO:teuthology.orchestra.run.smithi187.stdout:(88/128): abseil-cpp-20211102.0-3.el9.x86_64.rp 853 kB/s | 548 kB 00:00 2024-05-08T04:06:34.001 INFO:teuthology.orchestra.run.smithi187.stdout:(89/128): libarrow-doc-9.0.0-11.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-05-08T04:06:34.035 INFO:teuthology.orchestra.run.smithi187.stdout:(90/128): liboath-2.6.7-2.el9.x86_64.rpm 447 kB/s | 49 kB 00:00 2024-05-08T04:06:34.110 INFO:teuthology.orchestra.run.smithi187.stdout:(91/128): libunwind-1.6.2-1.el9.x86_64.rpm 618 kB/s | 67 kB 00:00 2024-05-08T04:06:34.152 INFO:teuthology.orchestra.run.smithi187.stdout:(92/128): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-05-08T04:06:34.302 INFO:teuthology.orchestra.run.smithi187.stdout:(93/128): python3-asyncssh-2.13.2-5.el9.noarch. 3.6 MB/s | 548 kB 00:00 2024-05-08T04:06:34.336 INFO:teuthology.orchestra.run.smithi187.stdout:(94/128): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.6 MB/s | 839 kB 00:00 2024-05-08T04:06:34.395 INFO:teuthology.orchestra.run.smithi187.stdout:(95/128): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 472 kB/s | 43 kB 00:00 2024-05-08T04:06:34.436 INFO:teuthology.orchestra.run.smithi187.stdout:(96/128): python3-cachetools-4.2.4-1.el9.noarch 321 kB/s | 32 kB 00:00 2024-05-08T04:06:34.478 INFO:teuthology.orchestra.run.smithi187.stdout:(97/128): python3-certifi-2021.10.8-2.el9.noarc 179 kB/s | 15 kB 00:00 2024-05-08T04:06:34.537 INFO:teuthology.orchestra.run.smithi187.stdout:(98/128): libarrow-9.0.0-11.el9.x86_64.rpm 6.3 MB/s | 4.4 MB 00:00 2024-05-08T04:06:34.562 INFO:teuthology.orchestra.run.smithi187.stdout:(99/128): python3-cheroot-8.6.0-4.el9.noarch.rp 1.3 MB/s | 172 kB 00:00 2024-05-08T04:06:34.604 INFO:teuthology.orchestra.run.smithi187.stdout:(100/128): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-05-08T04:06:34.654 INFO:teuthology.orchestra.run.smithi187.stdout:(101/128): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-05-08T04:06:34.705 INFO:teuthology.orchestra.run.smithi187.stdout:(102/128): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-05-08T04:06:34.789 INFO:teuthology.orchestra.run.smithi187.stdout:(103/128): python3-grpcio-1.46.7-10.el9.x86_64. 9.0 MB/s | 2.0 MB 00:00 2024-05-08T04:06:34.814 INFO:teuthology.orchestra.run.smithi187.stdout:(104/128): python3-jaraco-8.2.1-3.el9.noarch.rp 67 kB/s | 11 kB 00:00 2024-05-08T04:06:34.839 INFO:teuthology.orchestra.run.smithi187.stdout:(105/128): python3-jaraco-classes-3.2.1-5.el9.n 132 kB/s | 18 kB 00:00 2024-05-08T04:06:34.881 INFO:teuthology.orchestra.run.smithi187.stdout:(106/128): python3-jaraco-collections-3.0.0-8.e 252 kB/s | 23 kB 00:00 2024-05-08T04:06:34.888 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2024-05-08T04:06:34.888 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2024-05-08T04:06:34.906 INFO:teuthology.orchestra.run.smithi187.stdout:(107/128): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-05-08T04:06:34.932 INFO:teuthology.orchestra.run.smithi187.stdout:(108/128): python3-jaraco-text-3.2.0-6.el9.noar 213 kB/s | 20 kB 00:00 2024-05-08T04:06:35.015 INFO:teuthology.orchestra.run.smithi187.stdout:(109/128): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-05-08T04:06:35.040 INFO:teuthology.orchestra.run.smithi187.stdout:(110/128): python3-logutils-0.3.5-21.el9.noarch 346 kB/s | 46 kB 00:00 2024-05-08T04:06:35.066 INFO:teuthology.orchestra.run.smithi187.stdout:(111/128): python3-more-itertools-8.12.0-2.el9. 590 kB/s | 79 kB 00:00 2024-05-08T04:06:35.166 INFO:teuthology.orchestra.run.smithi187.stdout:(112/128): python3-natsort-7.1.1-5.el9.noarch.r 461 kB/s | 58 kB 00:00 2024-05-08T04:06:35.191 INFO:teuthology.orchestra.run.smithi187.stdout:(113/128): python3-portend-3.1.0-2.el9.noarch.r 131 kB/s | 16 kB 00:00 2024-05-08T04:06:35.216 INFO:teuthology.orchestra.run.smithi187.stdout:(114/128): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-05-08T04:06:35.267 INFO:teuthology.orchestra.run.smithi187.stdout:(115/128): python3-pyOpenSSL-21.0.0-1.el9.noarc 894 kB/s | 90 kB 00:00 2024-05-08T04:06:35.292 INFO:teuthology.orchestra.run.smithi187.stdout:(116/128): python3-repoze-lru-0.7-16.el9.noarch 306 kB/s | 31 kB 00:00 2024-05-08T04:06:35.317 INFO:teuthology.orchestra.run.smithi187.stdout:(117/128): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-05-08T04:06:35.376 INFO:teuthology.orchestra.run.smithi187.stdout:(118/128): python3-rsa-4.9-2.el9.noarch.rpm 543 kB/s | 59 kB 00:00 2024-05-08T04:06:35.418 INFO:teuthology.orchestra.run.smithi187.stdout:(119/128): python3-tempora-5.0.0-2.el9.noarch.r 284 kB/s | 36 kB 00:00 2024-05-08T04:06:35.443 INFO:teuthology.orchestra.run.smithi187.stdout:(120/128): python3-typing-extensions-4.4.0-2.el 405 kB/s | 51 kB 00:00 2024-05-08T04:06:35.494 INFO:teuthology.orchestra.run.smithi187.stdout:(121/128): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-05-08T04:06:35.552 INFO:teuthology.orchestra.run.smithi187.stdout:(122/128): python3-websocket-client-1.2.3-2.el9 670 kB/s | 90 kB 00:00 2024-05-08T04:06:35.645 INFO:teuthology.orchestra.run.smithi187.stdout:(123/128): python3-werkzeug-2.0.3-3.el9.1.noarc 2.1 MB/s | 427 kB 00:00 2024-05-08T04:06:35.669 INFO:teuthology.orchestra.run.smithi187.stdout:(124/128): python3-zc-lockfile-2.0-10.el9.noarc 113 kB/s | 20 kB 00:00 2024-05-08T04:06:35.695 INFO:teuthology.orchestra.run.smithi187.stdout:(125/128): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-05-08T04:06:35.787 INFO:teuthology.orchestra.run.smithi187.stdout:(126/128): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-08T04:06:36.037 INFO:teuthology.orchestra.run.smithi187.stdout:(127/128): librbd1-19.0.0-3512.g5ccf1c30.el9.x8 9.2 MB/s | 3.1 MB 00:00 2024-05-08T04:06:36.146 INFO:teuthology.orchestra.run.smithi187.stdout:(128/128): librados2-19.0.0-3512.g5ccf1c30.el9. 7.1 MB/s | 3.4 MB 00:00 2024-05-08T04:06:36.149 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2024-05-08T04:06:36.149 INFO:teuthology.orchestra.run.smithi187.stdout:Total 9.3 MB/s | 203 MB 00:21 2024-05-08T04:06:37.009 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2024-05-08T04:06:37.116 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2024-05-08T04:06:37.116 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2024-05-08T04:06:39.121 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2024-05-08T04:06:39.121 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2024-05-08T04:06:39.911 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2024-05-08T04:06:39.981 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/130 2024-05-08T04:06:40.006 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/130 2024-05-08T04:06:40.066 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/130 2024-05-08T04:06:40.405 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/130 2024-05-08T04:06:40.524 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : librados2-2:19.0.0-3512.g5ccf1c30.el9.x86_64 5/130 2024-05-08T04:06:40.646 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados2-2:19.0.0-3512.g5ccf1c30.el9.x86_64 5/130 2024-05-08T04:06:40.725 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephfs2-2:19.0.0-3512.g5ccf1c30.el9.x86_64 6/130 2024-05-08T04:06:40.824 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephfs2-2:19.0.0-3512.g5ccf1c30.el9.x86_64 6/130 2024-05-08T04:06:40.874 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rados-2:19.0.0-3512.g5ccf1c30.el9.x86_64 7/130 2024-05-08T04:06:40.905 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/130 2024-05-08T04:06:40.942 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/130 2024-05-08T04:06:40.969 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephsqlite-2:19.0.0-3512.g5ccf1c30.el9.x86_64 10/130 2024-05-08T04:06:41.072 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3512.g5ccf1c30.el9.x86_64 10/130 2024-05-08T04:06:41.095 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libradosstriper1-2:19.0.0-3512.g5ccf1c30.el9.x86 11/130 2024-05-08T04:06:41.142 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3512.g5ccf1c30.el9.x86 11/130 2024-05-08T04:06:41.163 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 12/130 2024-05-08T04:06:41.182 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 12/130 2024-05-08T04:06:41.364 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 13/130 2024-05-08T04:06:41.396 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 13/130 2024-05-08T04:06:41.521 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : re2-1:20211101-3.el9.x86_64 14/130 2024-05-08T04:06:41.561 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 15/130 2024-05-08T04:06:41.596 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : liboath-2.6.7-2.el9.x86_64 16/130 2024-05-08T04:06:41.661 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 17/130 2024-05-08T04:06:41.713 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 18/130 2024-05-08T04:06:41.755 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/130 2024-05-08T04:06:41.789 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 20/130 2024-05-08T04:06:41.816 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : lua-5.4.4-4.el9.x86_64 21/130 2024-05-08T04:06:41.885 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 22/130 2024-05-08T04:06:41.947 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 23/130 2024-05-08T04:06:41.983 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 24/130 2024-05-08T04:06:42.021 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 25/130 2024-05-08T04:06:42.098 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 26/130 2024-05-08T04:06:42.140 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ceph-common-2:19.0.0-3512.g5ccf1c30.el9. 27/130 2024-05-08T04:06:42.184 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ceph-argparse-2:19.0.0-3512.g5ccf1c30.el 28/130 2024-05-08T04:06:42.212 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cephfs-2:19.0.0-3512.g5ccf1c30.el9.x86_6 29/130 2024-05-08T04:06:42.283 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 30/130 2024-05-08T04:06:42.328 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : luarocks-3.9.2-1.el9.noarch 31/130 2024-05-08T04:06:42.402 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 32/130 2024-05-08T04:06:42.545 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 33/130 2024-05-08T04:06:42.596 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 34/130 2024-05-08T04:06:43.131 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rsa-4.9-2.el9.noarch 35/130 2024-05-08T04:06:43.218 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 36/130 2024-05-08T04:06:43.246 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 37/130 2024-05-08T04:06:43.281 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 38/130 2024-05-08T04:06:43.318 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librados-devel-2:19.0.0-3512.g5ccf1c30.el9.x86_6 39/130 2024-05-08T04:06:43.352 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 40/130 2024-05-08T04:06:43.444 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 41/130 2024-05-08T04:06:43.499 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 42/130 2024-05-08T04:06:43.564 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 43/130 2024-05-08T04:06:43.595 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 44/130 2024-05-08T04:06:43.631 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 45/130 2024-05-08T04:06:43.679 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 46/130 2024-05-08T04:06:43.725 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 47/130 2024-05-08T04:06:43.766 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 48/130 2024-05-08T04:06:43.903 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 49/130 2024-05-08T04:06:43.943 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 50/130 2024-05-08T04:06:43.984 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 51/130 2024-05-08T04:06:44.079 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 52/130 2024-05-08T04:06:44.126 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 53/130 2024-05-08T04:06:44.173 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 54/130 2024-05-08T04:06:44.199 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 55/130 2024-05-08T04:06:44.225 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 56/130 2024-05-08T04:06:44.251 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2024-05-08T04:06:44.315 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 57/130 2024-05-08T04:06:44.315 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/130 2024-05-08T04:06:44.340 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/130 2024-05-08T04:06:44.396 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/130 2024-05-08T04:06:44.443 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 58/130 2024-05-08T04:06:44.499 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 59/130 2024-05-08T04:06:44.537 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 60/130 2024-05-08T04:06:44.580 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : sqlite-3.34.1-7.el9.x86_64 61/130 2024-05-08T04:06:44.614 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 62/130 2024-05-08T04:06:44.644 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 63/130 2024-05-08T04:06:44.678 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 64/130 2024-05-08T04:06:44.712 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 65/130 2024-05-08T04:06:44.730 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/130 2024-05-08T04:06:44.753 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 66/130 2024-05-08T04:06:44.788 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 67/130 2024-05-08T04:06:44.849 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : librados2-2:19.0.0-3512.g5ccf1c30.el9.x86_64 5/130 2024-05-08T04:06:44.875 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 68/130 2024-05-08T04:06:44.927 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 69/130 2024-05-08T04:06:44.958 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 70/130 2024-05-08T04:06:44.971 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librados2-2:19.0.0-3512.g5ccf1c30.el9.x86_64 5/130 2024-05-08T04:06:45.005 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 71/130 2024-05-08T04:06:45.009 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephfs2-2:19.0.0-3512.g5ccf1c30.el9.x86_64 6/130 2024-05-08T04:06:45.093 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libcephfs2-2:19.0.0-3512.g5ccf1c30.el9.x86_64 6/130 2024-05-08T04:06:45.104 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-packaging-20.9-5.el9.noarch 72/130 2024-05-08T04:06:45.148 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rados-2:19.0.0-3512.g5ccf1c30.el9.x86_64 7/130 2024-05-08T04:06:45.168 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 73/130 2024-05-08T04:06:45.181 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/130 2024-05-08T04:06:45.216 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/130 2024-05-08T04:06:45.245 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephsqlite-2:19.0.0-3512.g5ccf1c30.el9.x86_64 10/130 2024-05-08T04:06:45.348 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3512.g5ccf1c30.el9.x86_64 10/130 2024-05-08T04:06:45.371 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libradosstriper1-2:19.0.0-3512.g5ccf1c30.el9.x86 11/130 2024-05-08T04:06:45.426 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3512.g5ccf1c30.el9.x86 11/130 2024-05-08T04:06:45.455 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 12/130 2024-05-08T04:06:45.476 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 12/130 2024-05-08T04:06:45.673 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 13/130 2024-05-08T04:06:45.705 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 13/130 2024-05-08T04:06:45.813 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : re2-1:20211101-3.el9.x86_64 14/130 2024-05-08T04:06:45.835 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 15/130 2024-05-08T04:06:45.879 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : liboath-2.6.7-2.el9.x86_64 16/130 2024-05-08T04:06:45.903 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/130 2024-05-08T04:06:45.949 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 75/130 2024-05-08T04:06:45.953 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 17/130 2024-05-08T04:06:45.982 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 76/130 2024-05-08T04:06:45.995 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 18/130 2024-05-08T04:06:46.038 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/130 2024-05-08T04:06:46.071 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 20/130 2024-05-08T04:06:46.083 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 77/130 2024-05-08T04:06:46.098 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : lua-5.4.4-4.el9.x86_64 21/130 2024-05-08T04:06:46.128 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 78/130 2024-05-08T04:06:46.168 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 22/130 2024-05-08T04:06:46.230 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 23/130 2024-05-08T04:06:46.266 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 24/130 2024-05-08T04:06:46.295 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openblas-0.3.21-2.el9.x86_64 79/130 2024-05-08T04:06:46.303 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 25/130 2024-05-08T04:06:46.322 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 80/130 2024-05-08T04:06:46.381 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 26/130 2024-05-08T04:06:46.400 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 81/130 2024-05-08T04:06:46.422 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-ceph-common-2:19.0.0-3512.g5ccf1c30.el9. 27/130 2024-05-08T04:06:46.465 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-ceph-argparse-2:19.0.0-3512.g5ccf1c30.el 28/130 2024-05-08T04:06:46.494 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cephfs-2:19.0.0-3512.g5ccf1c30.el9.x86_6 29/130 2024-05-08T04:06:46.565 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 30/130 2024-05-08T04:06:46.618 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : luarocks-3.9.2-1.el9.noarch 31/130 2024-05-08T04:06:46.693 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 32/130 2024-05-08T04:06:46.828 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 33/130 2024-05-08T04:06:46.878 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 34/130 2024-05-08T04:06:47.089 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 82/130 2024-05-08T04:06:47.221 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 83/130 2024-05-08T04:06:47.430 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rsa-4.9-2.el9.noarch 35/130 2024-05-08T04:06:47.516 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 36/130 2024-05-08T04:06:47.544 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 37/130 2024-05-08T04:06:47.579 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 38/130 2024-05-08T04:06:47.617 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librados-devel-2:19.0.0-3512.g5ccf1c30.el9.x86_6 39/130 2024-05-08T04:06:47.650 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 40/130 2024-05-08T04:06:47.746 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 41/130 2024-05-08T04:06:47.815 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 42/130 2024-05-08T04:06:47.888 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 43/130 2024-05-08T04:06:47.935 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 44/130 2024-05-08T04:06:47.971 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 45/130 2024-05-08T04:06:48.019 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 46/130 2024-05-08T04:06:48.064 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 47/130 2024-05-08T04:06:48.106 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 48/130 2024-05-08T04:06:48.252 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 49/130 2024-05-08T04:06:48.291 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 50/130 2024-05-08T04:06:48.333 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 51/130 2024-05-08T04:06:48.429 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 52/130 2024-05-08T04:06:48.514 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/130 2024-05-08T04:06:48.621 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 85/130 2024-05-08T04:06:48.649 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 53/130 2024-05-08T04:06:48.694 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 54/130 2024-05-08T04:06:48.722 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 55/130 2024-05-08T04:06:48.747 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 56/130 2024-05-08T04:06:48.860 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 57/130 2024-05-08T04:06:48.936 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 86/130 2024-05-08T04:06:48.975 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : librbd1-2:19.0.0-3512.g5ccf1c30.el9.x86_64 87/130 2024-05-08T04:06:48.992 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 58/130 2024-05-08T04:06:49.032 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 59/130 2024-05-08T04:06:49.064 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librbd1-2:19.0.0-3512.g5ccf1c30.el9.x86_64 87/130 2024-05-08T04:06:49.068 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 60/130 2024-05-08T04:06:49.096 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rbd-2:19.0.0-3512.g5ccf1c30.el9.x86_64 88/130 2024-05-08T04:06:49.111 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : sqlite-3.34.1-7.el9.x86_64 61/130 2024-05-08T04:06:49.132 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 89/130 2024-05-08T04:06:49.154 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 62/130 2024-05-08T04:06:49.183 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 63/130 2024-05-08T04:06:49.218 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 64/130 2024-05-08T04:06:49.259 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 65/130 2024-05-08T04:06:49.301 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 66/130 2024-05-08T04:06:49.335 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 67/130 2024-05-08T04:06:49.424 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 68/130 2024-05-08T04:06:49.474 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 69/130 2024-05-08T04:06:49.522 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 70/130 2024-05-08T04:06:49.568 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 71/130 2024-05-08T04:06:49.609 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 90/130 2024-05-08T04:06:49.636 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librgw2-2:19.0.0-3512.g5ccf1c30.el9.x86_64 91/130 2024-05-08T04:06:49.669 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-packaging-20.9-5.el9.noarch 72/130 2024-05-08T04:06:49.707 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librgw2-2:19.0.0-3512.g5ccf1c30.el9.x86_64 91/130 2024-05-08T04:06:49.724 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 73/130 2024-05-08T04:06:49.729 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rgw-2:19.0.0-3512.g5ccf1c30.el9.x86_64 92/130 2024-05-08T04:06:50.465 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/130 2024-05-08T04:06:50.613 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 75/130 2024-05-08T04:06:50.655 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 76/130 2024-05-08T04:06:50.756 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 77/130 2024-05-08T04:06:50.797 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 78/130 2024-05-08T04:06:50.973 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openblas-0.3.21-2.el9.x86_64 79/130 2024-05-08T04:06:51.003 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 80/130 2024-05-08T04:06:51.081 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 81/130 2024-05-08T04:06:51.754 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 82/130 2024-05-08T04:06:51.869 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 83/130 2024-05-08T04:06:52.021 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-common-2:19.0.0-3512.g5ccf1c30.el9.x86_64 93/130 2024-05-08T04:06:52.084 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-common-2:19.0.0-3512.g5ccf1c30.el9.x86_64 93/130 2024-05-08T04:06:52.638 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-common-2:19.0.0-3512.g5ccf1c30.el9.x86_64 93/130 2024-05-08T04:06:52.679 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-base-2:19.0.0-3512.g5ccf1c30.el9.x86_64 94/130 2024-05-08T04:06:52.773 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-base-2:19.0.0-3512.g5ccf1c30.el9.x86_64 94/130 2024-05-08T04:06:52.773 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T04:06:52.774 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T04:06:52.774 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:06:52.830 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-selinux-2:19.0.0-3512.g5ccf1c30.el9.x86_64 95/130 2024-05-08T04:06:53.153 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/130 2024-05-08T04:06:53.242 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 85/130 2024-05-08T04:06:53.553 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 86/130 2024-05-08T04:06:53.595 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : librbd1-2:19.0.0-3512.g5ccf1c30.el9.x86_64 87/130 2024-05-08T04:06:53.685 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librbd1-2:19.0.0-3512.g5ccf1c30.el9.x86_64 87/130 2024-05-08T04:06:53.717 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rbd-2:19.0.0-3512.g5ccf1c30.el9.x86_64 88/130 2024-05-08T04:06:53.762 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 89/130 2024-05-08T04:06:54.240 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 90/130 2024-05-08T04:06:54.273 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librgw2-2:19.0.0-3512.g5ccf1c30.el9.x86_64 91/130 2024-05-08T04:06:54.328 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librgw2-2:19.0.0-3512.g5ccf1c30.el9.x86_64 91/130 2024-05-08T04:06:54.358 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rgw-2:19.0.0-3512.g5ccf1c30.el9.x86_64 92/130 2024-05-08T04:06:56.607 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-common-2:19.0.0-3512.g5ccf1c30.el9.x86_64 93/130 2024-05-08T04:06:56.638 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-common-2:19.0.0-3512.g5ccf1c30.el9.x86_64 93/130 2024-05-08T04:06:57.187 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-common-2:19.0.0-3512.g5ccf1c30.el9.x86_64 93/130 2024-05-08T04:06:57.224 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-base-2:19.0.0-3512.g5ccf1c30.el9.x86_64 94/130 2024-05-08T04:06:57.308 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-base-2:19.0.0-3512.g5ccf1c30.el9.x86_64 94/130 2024-05-08T04:06:57.308 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T04:06:57.308 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T04:06:57.308 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:06:57.342 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-selinux-2:19.0.0-3512.g5ccf1c30.el9.x86_64 95/130 2024-05-08T04:07:04.321 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3512.g5ccf1c30.el9.x86_64 95/130 2024-05-08T04:07:04.321 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /sys 2024-05-08T04:07:04.321 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /proc 2024-05-08T04:07:04.321 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /mnt 2024-05-08T04:07:04.321 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /var/tmp 2024-05-08T04:07:04.321 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /home 2024-05-08T04:07:04.322 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /root 2024-05-08T04:07:04.322 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /tmp 2024-05-08T04:07:04.322 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:04.495 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-volume-2:19.0.0-3512.g5ccf1c30.el9.noarch 96/130 2024-05-08T04:07:04.512 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-volume-2:19.0.0-3512.g5ccf1c30.el9.noarch 96/130 2024-05-08T04:07:04.512 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T04:07:04.512 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T04:07:04.512 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:06.006 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-osd-2:19.0.0-3512.g5ccf1c30.el9.x86_64 97/130 2024-05-08T04:07:06.032 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-osd-2:19.0.0-3512.g5ccf1c30.el9.x86_64 97/130 2024-05-08T04:07:06.032 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T04:07:06.033 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T04:07:06.033 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T04:07:06.033 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T04:07:06.033 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:06.257 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mds-2:19.0.0-3512.g5ccf1c30.el9.x86_64 98/130 2024-05-08T04:07:06.280 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mds-2:19.0.0-3512.g5ccf1c30.el9.x86_64 98/130 2024-05-08T04:07:06.280 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T04:07:06.280 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T04:07:06.280 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T04:07:06.280 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T04:07:06.280 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:06.716 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mon-2:19.0.0-3512.g5ccf1c30.el9.x86_64 99/130 2024-05-08T04:07:06.739 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mon-2:19.0.0-3512.g5ccf1c30.el9.x86_64 99/130 2024-05-08T04:07:06.739 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T04:07:06.739 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T04:07:06.739 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T04:07:06.739 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T04:07:06.739 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:06.802 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ply-3.11-14.el9.noarch 100/130 2024-05-08T04:07:06.866 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 101/130 2024-05-08T04:07:07.052 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 102/130 2024-05-08T04:07:07.107 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 103/130 2024-05-08T04:07:07.176 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 104/130 2024-05-08T04:07:07.265 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 105/130 2024-05-08T04:07:07.405 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 106/130 2024-05-08T04:07:07.450 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 107/130 2024-05-08T04:07:07.481 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 108/130 2024-05-08T04:07:07.509 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : fuse-2.9.9-15.el9.x86_64 109/130 2024-05-08T04:07:07.937 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: cephadm-2:19.0.0-3512.g5ccf1c30.el9.noarch 110/130 2024-05-08T04:07:07.966 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephadm-2:19.0.0-3512.g5ccf1c30.el9.noarch 110/130 2024-05-08T04:07:07.998 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3512.g5ccf1c30.e 111/130 2024-05-08T04:07:08.060 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3512.g5ccf1c30. 112/130 2024-05-08T04:07:08.089 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3512.g5ccf1c30.el9.noa 113/130 2024-05-08T04:07:08.466 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3512.g5ccf1c30.el9.x86_64 95/130 2024-05-08T04:07:08.466 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /sys 2024-05-08T04:07:08.467 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /proc 2024-05-08T04:07:08.467 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /mnt 2024-05-08T04:07:08.467 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /var/tmp 2024-05-08T04:07:08.467 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /home 2024-05-08T04:07:08.467 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /root 2024-05-08T04:07:08.467 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /tmp 2024-05-08T04:07:08.467 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:08.636 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-volume-2:19.0.0-3512.g5ccf1c30.el9.noarch 96/130 2024-05-08T04:07:08.651 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-volume-2:19.0.0-3512.g5ccf1c30.el9.noarch 96/130 2024-05-08T04:07:08.651 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T04:07:08.652 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T04:07:08.652 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:09.123 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3512.g5ccf1c30.el9.noa 113/130 2024-05-08T04:07:09.157 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3512.g5ccf1c30.el9.n 114/130 2024-05-08T04:07:10.117 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3512.g5ccf1c30.el9.n 114/130 2024-05-08T04:07:10.129 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-osd-2:19.0.0-3512.g5ccf1c30.el9.x86_64 97/130 2024-05-08T04:07:10.141 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3512.g5cc 115/130 2024-05-08T04:07:10.157 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-osd-2:19.0.0-3512.g5ccf1c30.el9.x86_64 97/130 2024-05-08T04:07:10.157 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T04:07:10.157 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T04:07:10.157 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T04:07:10.157 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T04:07:10.158 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:10.252 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3512.g5cc 115/130 2024-05-08T04:07:10.389 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mds-2:19.0.0-3512.g5ccf1c30.el9.x86_64 98/130 2024-05-08T04:07:10.412 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mds-2:19.0.0-3512.g5ccf1c30.el9.x86_64 98/130 2024-05-08T04:07:10.413 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T04:07:10.413 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T04:07:10.413 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T04:07:10.413 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T04:07:10.413 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:10.420 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3512.g5ccf1c30.el 116/130 2024-05-08T04:07:10.452 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-2:19.0.0-3512.g5ccf1c30.el9.x86_64 117/130 2024-05-08T04:07:10.475 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3512.g5ccf1c30.el9.x86_64 117/130 2024-05-08T04:07:10.475 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T04:07:10.475 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T04:07:10.475 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T04:07:10.475 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T04:07:10.475 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:10.527 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-rook-2:19.0.0-3512.g5ccf1c30.el9.noarch 118/130 2024-05-08T04:07:10.539 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3512.g5ccf1c30.el9.noarch 118/130 2024-05-08T04:07:10.643 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-2:19.0.0-3512.g5ccf1c30.el9.x86_64 119/130 2024-05-08T04:07:10.866 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mon-2:19.0.0-3512.g5ccf1c30.el9.x86_64 99/130 2024-05-08T04:07:10.889 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mon-2:19.0.0-3512.g5ccf1c30.el9.x86_64 99/130 2024-05-08T04:07:10.889 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T04:07:10.889 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T04:07:10.890 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T04:07:10.890 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T04:07:10.890 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:10.977 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-ply-3.11-14.el9.noarch 100/130 2024-05-08T04:07:11.040 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 101/130 2024-05-08T04:07:11.226 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 102/130 2024-05-08T04:07:11.282 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 103/130 2024-05-08T04:07:11.359 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 104/130 2024-05-08T04:07:11.450 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 105/130 2024-05-08T04:07:11.565 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-fuse-2:19.0.0-3512.g5ccf1c30.el9.x86_64 120/130 2024-05-08T04:07:11.595 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 106/130 2024-05-08T04:07:11.595 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-radosgw-2:19.0.0-3512.g5ccf1c30.el9.x86_64 121/130 2024-05-08T04:07:11.619 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3512.g5ccf1c30.el9.x86_64 121/130 2024-05-08T04:07:11.619 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T04:07:11.619 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T04:07:11.619 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T04:07:11.619 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T04:07:11.619 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:11.632 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 107/130 2024-05-08T04:07:11.663 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3512.g5ccf1 122/130 2024-05-08T04:07:11.664 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 108/130 2024-05-08T04:07:11.685 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3512.g5ccf1 122/130 2024-05-08T04:07:11.685 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T04:07:11.686 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T04:07:11.686 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:11.700 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : fuse-2.9.9-15.el9.x86_64 109/130 2024-05-08T04:07:11.982 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-mirror-2:19.0.0-3512.g5ccf1c30.el9.x86_64 123/130 2024-05-08T04:07:12.005 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3512.g5ccf1c30.el9.x86_64 123/130 2024-05-08T04:07:12.005 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T04:07:12.005 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T04:07:12.005 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T04:07:12.005 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T04:07:12.005 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:12.120 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: cephadm-2:19.0.0-3512.g5ccf1c30.el9.noarch 110/130 2024-05-08T04:07:12.148 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephadm-2:19.0.0-3512.g5ccf1c30.el9.noarch 110/130 2024-05-08T04:07:12.181 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3512.g5ccf1c30.e 111/130 2024-05-08T04:07:12.242 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3512.g5ccf1c30. 112/130 2024-05-08T04:07:12.280 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3512.g5ccf1c30.el9.noa 113/130 2024-05-08T04:07:13.320 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3512.g5ccf1c30.el9.noa 113/130 2024-05-08T04:07:13.357 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3512.g5ccf1c30.el9.n 114/130 2024-05-08T04:07:14.318 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3512.g5ccf1c30.el9.n 114/130 2024-05-08T04:07:14.349 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3512.g5cc 115/130 2024-05-08T04:07:14.461 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3512.g5cc 115/130 2024-05-08T04:07:14.618 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3512.g5ccf1c30.el 116/130 2024-05-08T04:07:14.643 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-2:19.0.0-3512.g5ccf1c30.el9.x86_64 117/130 2024-05-08T04:07:14.667 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3512.g5ccf1c30.el9.x86_64 117/130 2024-05-08T04:07:14.667 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T04:07:14.667 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T04:07:14.667 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T04:07:14.667 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T04:07:14.667 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:14.710 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-rook-2:19.0.0-3512.g5ccf1c30.el9.noarch 118/130 2024-05-08T04:07:14.722 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3512.g5ccf1c30.el9.noarch 118/130 2024-05-08T04:07:14.829 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-2:19.0.0-3512.g5ccf1c30.el9.x86_64 119/130 2024-05-08T04:07:15.768 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-fuse-2:19.0.0-3512.g5ccf1c30.el9.x86_64 120/130 2024-05-08T04:07:15.795 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-radosgw-2:19.0.0-3512.g5ccf1c30.el9.x86_64 121/130 2024-05-08T04:07:15.819 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3512.g5ccf1c30.el9.x86_64 121/130 2024-05-08T04:07:15.819 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T04:07:15.819 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T04:07:15.819 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T04:07:15.819 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T04:07:15.819 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:15.862 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3512.g5ccf1 122/130 2024-05-08T04:07:15.885 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3512.g5ccf1 122/130 2024-05-08T04:07:15.885 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T04:07:15.885 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T04:07:15.885 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:16.196 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-mirror-2:19.0.0-3512.g5ccf1c30.el9.x86_64 123/130 2024-05-08T04:07:16.220 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3512.g5ccf1c30.el9.x86_64 123/130 2024-05-08T04:07:16.220 INFO:teuthology.orchestra.run.smithi187.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T04:07:16.220 INFO:teuthology.orchestra.run.smithi187.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T04:07:16.220 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T04:07:16.220 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T04:07:16.220 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:16.653 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-test-2:19.0.0-3512.g5ccf1c30.el9.x86_64 124/130 2024-05-08T04:07:16.694 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-fuse-2:19.0.0-3512.g5ccf1c30.el9.x86_64 125/130 2024-05-08T04:07:16.724 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-nbd-2:19.0.0-3512.g5ccf1c30.el9.x86_64 126/130 2024-05-08T04:07:16.750 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : sqlite-devel-3.34.1-7.el9.x86_64 127/130 2024-05-08T04:07:16.781 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephfs-devel-2:19.0.0-3512.g5ccf1c30.el9.x86_ 128/130 2024-05-08T04:07:16.782 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 129/130 2024-05-08T04:07:16.865 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 129/130 2024-05-08T04:07:16.865 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 130/130 2024-05-08T04:07:20.850 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-test-2:19.0.0-3512.g5ccf1c30.el9.x86_64 124/130 2024-05-08T04:07:20.892 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-fuse-2:19.0.0-3512.g5ccf1c30.el9.x86_64 125/130 2024-05-08T04:07:20.922 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-nbd-2:19.0.0-3512.g5ccf1c30.el9.x86_64 126/130 2024-05-08T04:07:20.948 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : sqlite-devel-3.34.1-7.el9.x86_64 127/130 2024-05-08T04:07:20.962 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephfs-devel-2:19.0.0-3512.g5ccf1c30.el9.x86_ 128/130 2024-05-08T04:07:20.962 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 129/130 2024-05-08T04:07:21.046 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 129/130 2024-05-08T04:07:21.046 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 130/130 2024-05-08T04:07:22.209 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 130/130 2024-05-08T04:07:22.209 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-2:19.0.0-3512.g5ccf1c30.el9.x86_64 1/130 2024-05-08T04:07:22.209 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-base-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2/130 2024-05-08T04:07:22.209 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-common-2:19.0.0-3512.g5ccf1c30.el9.x86_64 3/130 2024-05-08T04:07:22.209 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-fuse-2:19.0.0-3512.g5ccf1c30.el9.x86_64 4/130 2024-05-08T04:07:22.209 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3512.g5ccf1 5/130 2024-05-08T04:07:22.209 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mds-2:19.0.0-3512.g5ccf1c30.el9.x86_64 6/130 2024-05-08T04:07:22.210 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-2:19.0.0-3512.g5ccf1c30.el9.x86_64 7/130 2024-05-08T04:07:22.210 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mon-2:19.0.0-3512.g5ccf1c30.el9.x86_64 8/130 2024-05-08T04:07:22.210 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-osd-2:19.0.0-3512.g5ccf1c30.el9.x86_64 9/130 2024-05-08T04:07:22.210 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-radosgw-2:19.0.0-3512.g5ccf1c30.el9.x86_64 10/130 2024-05-08T04:07:22.210 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-selinux-2:19.0.0-3512.g5ccf1c30.el9.x86_64 11/130 2024-05-08T04:07:22.210 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-test-2:19.0.0-3512.g5ccf1c30.el9.x86_64 12/130 2024-05-08T04:07:22.211 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs-devel-2:19.0.0-3512.g5ccf1c30.el9.x86_ 13/130 2024-05-08T04:07:22.211 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs2-2:19.0.0-3512.g5ccf1c30.el9.x86_64 14/130 2024-05-08T04:07:22.211 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephsqlite-2:19.0.0-3512.g5ccf1c30.el9.x86_64 15/130 2024-05-08T04:07:22.211 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados-devel-2:19.0.0-3512.g5ccf1c30.el9.x86_6 16/130 2024-05-08T04:07:22.211 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libradosstriper1-2:19.0.0-3512.g5ccf1c30.el9.x86 17/130 2024-05-08T04:07:22.211 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librgw2-2:19.0.0-3512.g5ccf1c30.el9.x86_64 18/130 2024-05-08T04:07:22.211 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3512.g5ccf1c30.el 19/130 2024-05-08T04:07:22.211 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-common-2:19.0.0-3512.g5ccf1c30.el9. 20/130 2024-05-08T04:07:22.211 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cephfs-2:19.0.0-3512.g5ccf1c30.el9.x86_6 21/130 2024-05-08T04:07:22.211 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rados-2:19.0.0-3512.g5ccf1c30.el9.x86_64 22/130 2024-05-08T04:07:22.211 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rbd-2:19.0.0-3512.g5ccf1c30.el9.x86_64 23/130 2024-05-08T04:07:22.211 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rgw-2:19.0.0-3512.g5ccf1c30.el9.x86_64 24/130 2024-05-08T04:07:22.211 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-fuse-2:19.0.0-3512.g5ccf1c30.el9.x86_64 25/130 2024-05-08T04:07:22.211 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-mirror-2:19.0.0-3512.g5ccf1c30.el9.x86_64 26/130 2024-05-08T04:07:22.211 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-nbd-2:19.0.0-3512.g5ccf1c30.el9.x86_64 27/130 2024-05-08T04:07:22.221 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3512.g5ccf1c30. 28/130 2024-05-08T04:07:22.221 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3512.g5ccf1c30.el9.noa 29/130 2024-05-08T04:07:22.221 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3512.g5ccf1c30.el9.n 30/130 2024-05-08T04:07:22.221 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3512.g5cc 31/130 2024-05-08T04:07:22.221 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3512.g5ccf1c30.el 32/130 2024-05-08T04:07:22.222 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3512.g5ccf1c30.el9.noarch 33/130 2024-05-08T04:07:22.222 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3512.g5ccf1c30.e 34/130 2024-05-08T04:07:22.222 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-volume-2:19.0.0-3512.g5ccf1c30.el9.noarch 35/130 2024-05-08T04:07:22.222 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephadm-2:19.0.0-3512.g5ccf1c30.el9.noarch 36/130 2024-05-08T04:07:22.222 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 37/130 2024-05-08T04:07:22.222 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 38/130 2024-05-08T04:07:22.222 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 39/130 2024-05-08T04:07:22.222 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 40/130 2024-05-08T04:07:22.222 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 41/130 2024-05-08T04:07:22.222 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ply-3.11-14.el9.noarch 42/130 2024-05-08T04:07:22.223 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 43/130 2024-05-08T04:07:22.223 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 44/130 2024-05-08T04:07:22.223 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 45/130 2024-05-08T04:07:22.223 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 46/130 2024-05-08T04:07:22.223 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 47/130 2024-05-08T04:07:22.223 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 48/130 2024-05-08T04:07:22.223 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 49/130 2024-05-08T04:07:22.223 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 50/130 2024-05-08T04:07:22.223 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 51/130 2024-05-08T04:07:22.223 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 52/130 2024-05-08T04:07:22.223 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/130 2024-05-08T04:07:22.223 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/130 2024-05-08T04:07:22.223 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 55/130 2024-05-08T04:07:22.223 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 56/130 2024-05-08T04:07:22.223 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 57/130 2024-05-08T04:07:22.224 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 58/130 2024-05-08T04:07:22.224 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 59/130 2024-05-08T04:07:22.224 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 60/130 2024-05-08T04:07:22.224 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/130 2024-05-08T04:07:22.224 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/130 2024-05-08T04:07:22.224 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/130 2024-05-08T04:07:22.224 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/130 2024-05-08T04:07:22.224 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/130 2024-05-08T04:07:22.224 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/130 2024-05-08T04:07:22.224 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/130 2024-05-08T04:07:22.224 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/130 2024-05-08T04:07:22.224 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/130 2024-05-08T04:07:22.225 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/130 2024-05-08T04:07:22.225 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/130 2024-05-08T04:07:22.225 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/130 2024-05-08T04:07:22.225 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/130 2024-05-08T04:07:22.225 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/130 2024-05-08T04:07:22.225 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/130 2024-05-08T04:07:22.225 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/130 2024-05-08T04:07:22.225 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 77/130 2024-05-08T04:07:22.225 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 78/130 2024-05-08T04:07:22.225 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 79/130 2024-05-08T04:07:22.225 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 80/130 2024-05-08T04:07:22.225 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : sqlite-3.34.1-7.el9.x86_64 81/130 2024-05-08T04:07:22.226 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : sqlite-devel-3.34.1-7.el9.x86_64 82/130 2024-05-08T04:07:22.226 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 83/130 2024-05-08T04:07:22.226 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 84/130 2024-05-08T04:07:22.226 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 85/130 2024-05-08T04:07:22.226 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 86/130 2024-05-08T04:07:22.226 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 87/130 2024-05-08T04:07:22.226 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 88/130 2024-05-08T04:07:22.226 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 89/130 2024-05-08T04:07:22.226 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 90/130 2024-05-08T04:07:22.226 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 91/130 2024-05-08T04:07:22.226 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 92/130 2024-05-08T04:07:22.226 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 93/130 2024-05-08T04:07:22.227 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 94/130 2024-05-08T04:07:22.227 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 95/130 2024-05-08T04:07:22.227 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 96/130 2024-05-08T04:07:22.227 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 97/130 2024-05-08T04:07:22.227 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 98/130 2024-05-08T04:07:22.227 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 99/130 2024-05-08T04:07:22.227 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 100/130 2024-05-08T04:07:22.227 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 101/130 2024-05-08T04:07:22.227 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 102/130 2024-05-08T04:07:22.227 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/130 2024-05-08T04:07:22.227 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 104/130 2024-05-08T04:07:22.227 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 105/130 2024-05-08T04:07:22.227 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 106/130 2024-05-08T04:07:22.228 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 107/130 2024-05-08T04:07:22.228 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 108/130 2024-05-08T04:07:22.228 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 109/130 2024-05-08T04:07:22.228 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 110/130 2024-05-08T04:07:22.228 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 111/130 2024-05-08T04:07:22.228 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 112/130 2024-05-08T04:07:22.228 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 113/130 2024-05-08T04:07:22.228 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 114/130 2024-05-08T04:07:22.228 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/130 2024-05-08T04:07:22.228 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 116/130 2024-05-08T04:07:22.228 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 117/130 2024-05-08T04:07:22.228 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 118/130 2024-05-08T04:07:22.228 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 119/130 2024-05-08T04:07:22.228 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 120/130 2024-05-08T04:07:22.228 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 121/130 2024-05-08T04:07:22.229 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 122/130 2024-05-08T04:07:22.229 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 123/130 2024-05-08T04:07:22.229 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 124/130 2024-05-08T04:07:22.229 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : re2-1:20211101-3.el9.x86_64 125/130 2024-05-08T04:07:22.229 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 126/130 2024-05-08T04:07:22.229 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados2-2:19.0.0-3512.g5ccf1c30.el9.x86_64 127/130 2024-05-08T04:07:22.229 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 128/130 2024-05-08T04:07:22.229 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librbd1-2:19.0.0-3512.g5ccf1c30.el9.x86_64 129/130 2024-05-08T04:07:24.342 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 130/130 2024-05-08T04:07:24.343 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:24.343 INFO:teuthology.orchestra.run.smithi170.stdout:Upgraded: 2024-05-08T04:07:24.343 INFO:teuthology.orchestra.run.smithi170.stdout: librados2-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:24.343 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:24.343 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2024-05-08T04:07:24.343 INFO:teuthology.orchestra.run.smithi170.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T04:07:24.343 INFO:teuthology.orchestra.run.smithi170.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T04:07:24.343 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:24.343 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:24.344 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:24.344 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:24.344 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards-2:19.0.0-3512.g5ccf1c30.el9.noarch 2024-05-08T04:07:24.344 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:24.344 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:24.344 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:24.344 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm-2:19.0.0-3512.g5ccf1c30.el9.noarch 2024-05-08T04:07:24.345 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard-2:19.0.0-3512.g5ccf1c30.el9.noarch 2024-05-08T04:07:24.345 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3512.g5ccf1c30.el9.noarch 2024-05-08T04:07:24.345 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core-2:19.0.0-3512.g5ccf1c30.el9.noarch 2024-05-08T04:07:24.345 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook-2:19.0.0-3512.g5ccf1c30.el9.noarch 2024-05-08T04:07:24.345 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:24.345 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:24.345 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-prometheus-alerts-2:19.0.0-3512.g5ccf1c30.el9.noarch 2024-05-08T04:07:24.345 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:24.345 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:24.345 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:24.345 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-volume-2:19.0.0-3512.g5ccf1c30.el9.noarch 2024-05-08T04:07:24.345 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm-2:19.0.0-3512.g5ccf1c30.el9.noarch 2024-05-08T04:07:24.346 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T04:07:24.346 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T04:07:24.346 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T04:07:24.346 INFO:teuthology.orchestra.run.smithi170.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-08T04:07:24.346 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T04:07:24.346 INFO:teuthology.orchestra.run.smithi170.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T04:07:24.346 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T04:07:24.346 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T04:07:24.346 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:24.346 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:24.346 INFO:teuthology.orchestra.run.smithi170.stdout: libcephsqlite-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:24.346 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T04:07:24.346 INFO:teuthology.orchestra.run.smithi170.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T04:07:24.347 INFO:teuthology.orchestra.run.smithi170.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T04:07:24.347 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T04:07:24.347 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T04:07:24.347 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:24.347 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:24.347 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T04:07:24.347 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:24.347 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T04:07:24.347 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T04:07:24.347 INFO:teuthology.orchestra.run.smithi170.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T04:07:24.347 INFO:teuthology.orchestra.run.smithi170.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T04:07:24.348 INFO:teuthology.orchestra.run.smithi170.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T04:07:24.348 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T04:07:24.348 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T04:07:24.348 INFO:teuthology.orchestra.run.smithi170.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T04:07:24.348 INFO:teuthology.orchestra.run.smithi170.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T04:07:24.348 INFO:teuthology.orchestra.run.smithi170.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T04:07:24.348 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T04:07:24.348 INFO:teuthology.orchestra.run.smithi170.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T04:07:24.348 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T04:07:24.348 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T04:07:24.348 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:24.348 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:24.349 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:24.349 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T04:07:24.349 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T04:07:24.349 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T04:07:24.349 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T04:07:24.349 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T04:07:24.349 INFO:teuthology.orchestra.run.smithi170.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T04:07:24.349 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T04:07:24.349 INFO:teuthology.orchestra.run.smithi170.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T04:07:24.349 INFO:teuthology.orchestra.run.smithi170.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T04:07:24.349 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T04:07:24.349 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T04:07:24.349 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T04:07:24.349 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T04:07:24.349 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T04:07:24.350 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T04:07:24.350 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T04:07:24.350 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T04:07:24.350 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T04:07:24.350 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T04:07:24.350 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T04:07:24.350 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T04:07:24.350 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T04:07:24.350 INFO:teuthology.orchestra.run.smithi170.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T04:07:24.350 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T04:07:24.350 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T04:07:24.350 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T04:07:24.351 INFO:teuthology.orchestra.run.smithi170.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T04:07:24.351 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T04:07:24.351 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T04:07:24.351 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T04:07:24.351 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T04:07:24.351 INFO:teuthology.orchestra.run.smithi170.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T04:07:24.351 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T04:07:24.351 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T04:07:24.351 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T04:07:24.351 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T04:07:24.351 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T04:07:24.351 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:24.352 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:24.352 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T04:07:24.352 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T04:07:24.352 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T04:07:24.352 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:24.352 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T04:07:24.352 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T04:07:24.352 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T04:07:24.352 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T04:07:24.352 INFO:teuthology.orchestra.run.smithi170.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T04:07:24.352 INFO:teuthology.orchestra.run.smithi170.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T04:07:24.352 INFO:teuthology.orchestra.run.smithi170.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T04:07:24.352 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T04:07:24.352 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T04:07:24.352 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T04:07:24.352 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T04:07:24.353 INFO:teuthology.orchestra.run.smithi170.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T04:07:24.353 INFO:teuthology.orchestra.run.smithi170.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T04:07:24.353 INFO:teuthology.orchestra.run.smithi170.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T04:07:24.353 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:24.353 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:24.353 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:24.353 INFO:teuthology.orchestra.run.smithi170.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T04:07:24.353 INFO:teuthology.orchestra.run.smithi170.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T04:07:24.353 INFO:teuthology.orchestra.run.smithi170.stdout: sqlite-3.34.1-7.el9.x86_64 2024-05-08T04:07:24.353 INFO:teuthology.orchestra.run.smithi170.stdout: sqlite-devel-3.34.1-7.el9.x86_64 2024-05-08T04:07:24.353 INFO:teuthology.orchestra.run.smithi170.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T04:07:24.354 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T04:07:24.354 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:24.354 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-05-08T04:07:24.697 DEBUG:teuthology.parallel:result is None 2024-05-08T04:07:26.442 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 130/130 2024-05-08T04:07:26.443 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-2:19.0.0-3512.g5ccf1c30.el9.x86_64 1/130 2024-05-08T04:07:26.443 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-base-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2/130 2024-05-08T04:07:26.443 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-common-2:19.0.0-3512.g5ccf1c30.el9.x86_64 3/130 2024-05-08T04:07:26.443 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-fuse-2:19.0.0-3512.g5ccf1c30.el9.x86_64 4/130 2024-05-08T04:07:26.443 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3512.g5ccf1 5/130 2024-05-08T04:07:26.443 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mds-2:19.0.0-3512.g5ccf1c30.el9.x86_64 6/130 2024-05-08T04:07:26.443 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-2:19.0.0-3512.g5ccf1c30.el9.x86_64 7/130 2024-05-08T04:07:26.443 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mon-2:19.0.0-3512.g5ccf1c30.el9.x86_64 8/130 2024-05-08T04:07:26.443 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-osd-2:19.0.0-3512.g5ccf1c30.el9.x86_64 9/130 2024-05-08T04:07:26.443 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-radosgw-2:19.0.0-3512.g5ccf1c30.el9.x86_64 10/130 2024-05-08T04:07:26.443 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-selinux-2:19.0.0-3512.g5ccf1c30.el9.x86_64 11/130 2024-05-08T04:07:26.443 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-test-2:19.0.0-3512.g5ccf1c30.el9.x86_64 12/130 2024-05-08T04:07:26.443 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephfs-devel-2:19.0.0-3512.g5ccf1c30.el9.x86_ 13/130 2024-05-08T04:07:26.444 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephfs2-2:19.0.0-3512.g5ccf1c30.el9.x86_64 14/130 2024-05-08T04:07:26.444 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephsqlite-2:19.0.0-3512.g5ccf1c30.el9.x86_64 15/130 2024-05-08T04:07:26.444 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados-devel-2:19.0.0-3512.g5ccf1c30.el9.x86_6 16/130 2024-05-08T04:07:26.444 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libradosstriper1-2:19.0.0-3512.g5ccf1c30.el9.x86 17/130 2024-05-08T04:07:26.444 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librgw2-2:19.0.0-3512.g5ccf1c30.el9.x86_64 18/130 2024-05-08T04:07:26.444 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3512.g5ccf1c30.el 19/130 2024-05-08T04:07:26.444 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-ceph-common-2:19.0.0-3512.g5ccf1c30.el9. 20/130 2024-05-08T04:07:26.444 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cephfs-2:19.0.0-3512.g5ccf1c30.el9.x86_6 21/130 2024-05-08T04:07:26.444 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rados-2:19.0.0-3512.g5ccf1c30.el9.x86_64 22/130 2024-05-08T04:07:26.444 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rbd-2:19.0.0-3512.g5ccf1c30.el9.x86_64 23/130 2024-05-08T04:07:26.444 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rgw-2:19.0.0-3512.g5ccf1c30.el9.x86_64 24/130 2024-05-08T04:07:26.445 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-fuse-2:19.0.0-3512.g5ccf1c30.el9.x86_64 25/130 2024-05-08T04:07:26.445 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-mirror-2:19.0.0-3512.g5ccf1c30.el9.x86_64 26/130 2024-05-08T04:07:26.445 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-nbd-2:19.0.0-3512.g5ccf1c30.el9.x86_64 27/130 2024-05-08T04:07:26.445 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3512.g5ccf1c30. 28/130 2024-05-08T04:07:26.445 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3512.g5ccf1c30.el9.noa 29/130 2024-05-08T04:07:26.445 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3512.g5ccf1c30.el9.n 30/130 2024-05-08T04:07:26.446 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3512.g5cc 31/130 2024-05-08T04:07:26.446 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3512.g5ccf1c30.el 32/130 2024-05-08T04:07:26.446 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3512.g5ccf1c30.el9.noarch 33/130 2024-05-08T04:07:26.446 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3512.g5ccf1c30.e 34/130 2024-05-08T04:07:26.446 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-volume-2:19.0.0-3512.g5ccf1c30.el9.noarch 35/130 2024-05-08T04:07:26.447 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephadm-2:19.0.0-3512.g5ccf1c30.el9.noarch 36/130 2024-05-08T04:07:26.447 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 37/130 2024-05-08T04:07:26.447 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 38/130 2024-05-08T04:07:26.447 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 39/130 2024-05-08T04:07:26.447 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 40/130 2024-05-08T04:07:26.447 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 41/130 2024-05-08T04:07:26.448 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-ply-3.11-14.el9.noarch 42/130 2024-05-08T04:07:26.448 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 43/130 2024-05-08T04:07:26.448 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 44/130 2024-05-08T04:07:26.448 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 45/130 2024-05-08T04:07:26.448 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 46/130 2024-05-08T04:07:26.448 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 47/130 2024-05-08T04:07:26.449 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 48/130 2024-05-08T04:07:26.449 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 49/130 2024-05-08T04:07:26.449 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 50/130 2024-05-08T04:07:26.449 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 51/130 2024-05-08T04:07:26.449 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 52/130 2024-05-08T04:07:26.449 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/130 2024-05-08T04:07:26.450 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/130 2024-05-08T04:07:26.450 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 55/130 2024-05-08T04:07:26.450 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 56/130 2024-05-08T04:07:26.450 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 57/130 2024-05-08T04:07:26.450 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 58/130 2024-05-08T04:07:26.451 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 59/130 2024-05-08T04:07:26.451 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 60/130 2024-05-08T04:07:26.451 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/130 2024-05-08T04:07:26.451 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/130 2024-05-08T04:07:26.451 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/130 2024-05-08T04:07:26.451 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/130 2024-05-08T04:07:26.451 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/130 2024-05-08T04:07:26.452 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/130 2024-05-08T04:07:26.452 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/130 2024-05-08T04:07:26.452 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/130 2024-05-08T04:07:26.452 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/130 2024-05-08T04:07:26.452 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/130 2024-05-08T04:07:26.452 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/130 2024-05-08T04:07:26.452 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/130 2024-05-08T04:07:26.453 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/130 2024-05-08T04:07:26.453 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/130 2024-05-08T04:07:26.453 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/130 2024-05-08T04:07:26.453 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/130 2024-05-08T04:07:26.453 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 77/130 2024-05-08T04:07:26.453 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 78/130 2024-05-08T04:07:26.453 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 79/130 2024-05-08T04:07:26.454 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 80/130 2024-05-08T04:07:26.454 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : sqlite-3.34.1-7.el9.x86_64 81/130 2024-05-08T04:07:26.454 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : sqlite-devel-3.34.1-7.el9.x86_64 82/130 2024-05-08T04:07:26.454 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 83/130 2024-05-08T04:07:26.454 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 84/130 2024-05-08T04:07:26.454 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 85/130 2024-05-08T04:07:26.454 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 86/130 2024-05-08T04:07:26.455 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 87/130 2024-05-08T04:07:26.455 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 88/130 2024-05-08T04:07:26.455 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 89/130 2024-05-08T04:07:26.455 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 90/130 2024-05-08T04:07:26.455 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 91/130 2024-05-08T04:07:26.455 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 92/130 2024-05-08T04:07:26.455 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 93/130 2024-05-08T04:07:26.455 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 94/130 2024-05-08T04:07:26.456 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 95/130 2024-05-08T04:07:26.456 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 96/130 2024-05-08T04:07:26.456 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 97/130 2024-05-08T04:07:26.456 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 98/130 2024-05-08T04:07:26.457 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 99/130 2024-05-08T04:07:26.457 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 100/130 2024-05-08T04:07:26.457 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 101/130 2024-05-08T04:07:26.457 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 102/130 2024-05-08T04:07:26.457 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/130 2024-05-08T04:07:26.457 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 104/130 2024-05-08T04:07:26.458 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 105/130 2024-05-08T04:07:26.458 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 106/130 2024-05-08T04:07:26.458 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 107/130 2024-05-08T04:07:26.458 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 108/130 2024-05-08T04:07:26.458 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 109/130 2024-05-08T04:07:26.459 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 110/130 2024-05-08T04:07:26.459 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 111/130 2024-05-08T04:07:26.459 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 112/130 2024-05-08T04:07:26.459 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 113/130 2024-05-08T04:07:26.459 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 114/130 2024-05-08T04:07:26.460 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/130 2024-05-08T04:07:26.460 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 116/130 2024-05-08T04:07:26.460 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 117/130 2024-05-08T04:07:26.460 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 118/130 2024-05-08T04:07:26.460 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 119/130 2024-05-08T04:07:26.460 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 120/130 2024-05-08T04:07:26.461 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 121/130 2024-05-08T04:07:26.461 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 122/130 2024-05-08T04:07:26.461 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 123/130 2024-05-08T04:07:26.461 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 124/130 2024-05-08T04:07:26.461 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : re2-1:20211101-3.el9.x86_64 125/130 2024-05-08T04:07:26.461 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 126/130 2024-05-08T04:07:26.461 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados2-2:19.0.0-3512.g5ccf1c30.el9.x86_64 127/130 2024-05-08T04:07:26.462 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 128/130 2024-05-08T04:07:26.462 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librbd1-2:19.0.0-3512.g5ccf1c30.el9.x86_64 129/130 2024-05-08T04:07:28.661 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 130/130 2024-05-08T04:07:28.662 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:28.662 INFO:teuthology.orchestra.run.smithi187.stdout:Upgraded: 2024-05-08T04:07:28.662 INFO:teuthology.orchestra.run.smithi187.stdout: librados2-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:28.662 INFO:teuthology.orchestra.run.smithi187.stdout: librbd1-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:28.662 INFO:teuthology.orchestra.run.smithi187.stdout:Installed: 2024-05-08T04:07:28.662 INFO:teuthology.orchestra.run.smithi187.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T04:07:28.662 INFO:teuthology.orchestra.run.smithi187.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T04:07:28.662 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:28.662 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-base-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:28.662 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-common-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:28.662 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-fuse-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:28.662 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-grafana-dashboards-2:19.0.0-3512.g5ccf1c30.el9.noarch 2024-05-08T04:07:28.662 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-immutable-object-cache-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:28.663 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mds-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:28.663 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:28.663 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-cephadm-2:19.0.0-3512.g5ccf1c30.el9.noarch 2024-05-08T04:07:28.663 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-dashboard-2:19.0.0-3512.g5ccf1c30.el9.noarch 2024-05-08T04:07:28.663 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3512.g5ccf1c30.el9.noarch 2024-05-08T04:07:28.663 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-modules-core-2:19.0.0-3512.g5ccf1c30.el9.noarch 2024-05-08T04:07:28.663 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-rook-2:19.0.0-3512.g5ccf1c30.el9.noarch 2024-05-08T04:07:28.663 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mon-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:28.663 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-osd-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:28.663 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-prometheus-alerts-2:19.0.0-3512.g5ccf1c30.el9.noarch 2024-05-08T04:07:28.663 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-radosgw-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:28.664 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-selinux-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:28.664 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-test-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:28.664 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-volume-2:19.0.0-3512.g5ccf1c30.el9.noarch 2024-05-08T04:07:28.664 INFO:teuthology.orchestra.run.smithi187.stdout: cephadm-2:19.0.0-3512.g5ccf1c30.el9.noarch 2024-05-08T04:07:28.664 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T04:07:28.664 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T04:07:28.664 INFO:teuthology.orchestra.run.smithi187.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T04:07:28.664 INFO:teuthology.orchestra.run.smithi187.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-08T04:07:28.664 INFO:teuthology.orchestra.run.smithi187.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T04:07:28.664 INFO:teuthology.orchestra.run.smithi187.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T04:07:28.664 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T04:07:28.664 INFO:teuthology.orchestra.run.smithi187.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T04:07:28.664 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs-devel-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:28.664 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs2-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:28.665 INFO:teuthology.orchestra.run.smithi187.stdout: libcephsqlite-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:28.665 INFO:teuthology.orchestra.run.smithi187.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T04:07:28.665 INFO:teuthology.orchestra.run.smithi187.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T04:07:28.665 INFO:teuthology.orchestra.run.smithi187.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T04:07:28.665 INFO:teuthology.orchestra.run.smithi187.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T04:07:28.665 INFO:teuthology.orchestra.run.smithi187.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T04:07:28.665 INFO:teuthology.orchestra.run.smithi187.stdout: librados-devel-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:28.665 INFO:teuthology.orchestra.run.smithi187.stdout: libradosstriper1-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:28.665 INFO:teuthology.orchestra.run.smithi187.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T04:07:28.665 INFO:teuthology.orchestra.run.smithi187.stdout: librgw2-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:28.665 INFO:teuthology.orchestra.run.smithi187.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T04:07:28.665 INFO:teuthology.orchestra.run.smithi187.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T04:07:28.665 INFO:teuthology.orchestra.run.smithi187.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T04:07:28.666 INFO:teuthology.orchestra.run.smithi187.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T04:07:28.666 INFO:teuthology.orchestra.run.smithi187.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T04:07:28.666 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T04:07:28.666 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T04:07:28.666 INFO:teuthology.orchestra.run.smithi187.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T04:07:28.666 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T04:07:28.666 INFO:teuthology.orchestra.run.smithi187.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T04:07:28.666 INFO:teuthology.orchestra.run.smithi187.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T04:07:28.666 INFO:teuthology.orchestra.run.smithi187.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T04:07:28.666 INFO:teuthology.orchestra.run.smithi187.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T04:07:28.666 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T04:07:28.666 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-argparse-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:28.666 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-common-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:28.666 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cephfs-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:28.667 INFO:teuthology.orchestra.run.smithi187.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T04:07:28.667 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T04:07:28.667 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T04:07:28.667 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T04:07:28.667 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T04:07:28.667 INFO:teuthology.orchestra.run.smithi187.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T04:07:28.667 INFO:teuthology.orchestra.run.smithi187.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T04:07:28.667 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T04:07:28.667 INFO:teuthology.orchestra.run.smithi187.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T04:07:28.667 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T04:07:28.667 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T04:07:28.667 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T04:07:28.667 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T04:07:28.667 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T04:07:28.668 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T04:07:28.668 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T04:07:28.668 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T04:07:28.668 INFO:teuthology.orchestra.run.smithi187.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T04:07:28.668 INFO:teuthology.orchestra.run.smithi187.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T04:07:28.668 INFO:teuthology.orchestra.run.smithi187.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T04:07:28.668 INFO:teuthology.orchestra.run.smithi187.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T04:07:28.668 INFO:teuthology.orchestra.run.smithi187.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T04:07:28.668 INFO:teuthology.orchestra.run.smithi187.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T04:07:28.668 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T04:07:28.668 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T04:07:28.668 INFO:teuthology.orchestra.run.smithi187.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T04:07:28.668 INFO:teuthology.orchestra.run.smithi187.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T04:07:28.668 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T04:07:28.669 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T04:07:28.669 INFO:teuthology.orchestra.run.smithi187.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T04:07:28.669 INFO:teuthology.orchestra.run.smithi187.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T04:07:28.669 INFO:teuthology.orchestra.run.smithi187.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T04:07:28.669 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T04:07:28.669 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T04:07:28.669 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T04:07:28.669 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T04:07:28.669 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T04:07:28.669 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rados-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:28.669 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rbd-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:28.669 INFO:teuthology.orchestra.run.smithi187.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T04:07:28.669 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T04:07:28.669 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T04:07:28.670 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rgw-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:28.670 INFO:teuthology.orchestra.run.smithi187.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T04:07:28.670 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T04:07:28.670 INFO:teuthology.orchestra.run.smithi187.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T04:07:28.670 INFO:teuthology.orchestra.run.smithi187.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T04:07:28.670 INFO:teuthology.orchestra.run.smithi187.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T04:07:28.670 INFO:teuthology.orchestra.run.smithi187.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T04:07:28.670 INFO:teuthology.orchestra.run.smithi187.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T04:07:28.670 INFO:teuthology.orchestra.run.smithi187.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T04:07:28.670 INFO:teuthology.orchestra.run.smithi187.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T04:07:28.670 INFO:teuthology.orchestra.run.smithi187.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T04:07:28.670 INFO:teuthology.orchestra.run.smithi187.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T04:07:28.670 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T04:07:28.671 INFO:teuthology.orchestra.run.smithi187.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T04:07:28.671 INFO:teuthology.orchestra.run.smithi187.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T04:07:28.671 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-fuse-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:28.671 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-mirror-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:28.671 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-nbd-2:19.0.0-3512.g5ccf1c30.el9.x86_64 2024-05-08T04:07:28.671 INFO:teuthology.orchestra.run.smithi187.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T04:07:28.671 INFO:teuthology.orchestra.run.smithi187.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T04:07:28.671 INFO:teuthology.orchestra.run.smithi187.stdout: sqlite-3.34.1-7.el9.x86_64 2024-05-08T04:07:28.671 INFO:teuthology.orchestra.run.smithi187.stdout: sqlite-devel-3.34.1-7.el9.x86_64 2024-05-08T04:07:28.672 INFO:teuthology.orchestra.run.smithi187.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T04:07:28.672 INFO:teuthology.orchestra.run.smithi187.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T04:07:28.672 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:28.672 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2024-05-08T04:07:28.987 DEBUG:teuthology.parallel:result is None 2024-05-08T04:07:28.989 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5ccf1c30cf724d5371f55439b3b3a90cb994fae2 2024-05-08T04:07:29.213 DEBUG:teuthology.orchestra.run.smithi170:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T04:07:29.240 INFO:teuthology.orchestra.run.smithi170.stdout:19.0.0-3512.g5ccf1c30.el9 2024-05-08T04:07:29.241 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3512.g5ccf1c30.el9 2024-05-08T04:07:29.241 INFO:teuthology.task.install:The correct ceph version 19.0.0-3512.g5ccf1c30 is installed. 2024-05-08T04:07:29.243 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5ccf1c30cf724d5371f55439b3b3a90cb994fae2 2024-05-08T04:07:29.387 DEBUG:teuthology.orchestra.run.smithi187:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T04:07:29.410 INFO:teuthology.orchestra.run.smithi187.stdout:19.0.0-3512.g5ccf1c30.el9 2024-05-08T04:07:29.410 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3512.g5ccf1c30.el9 2024-05-08T04:07:29.411 INFO:teuthology.task.install:The correct ceph version 19.0.0-3512.g5ccf1c30 is installed. 2024-05-08T04:07:29.412 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-08T04:07:29.413 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-08T04:07:29.413 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T04:07:29.444 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-08T04:07:29.445 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T04:07:29.478 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-08T04:07:29.478 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-08T04:07:29.479 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T04:07:29.513 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T04:07:29.584 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-08T04:07:29.584 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T04:07:29.611 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T04:07:29.677 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-08T04:07:29.677 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-08T04:07:29.677 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T04:07:29.704 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T04:07:29.771 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-08T04:07:29.771 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T04:07:29.796 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T04:07:29.860 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-08T04:07:29.860 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-08T04:07:29.860 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T04:07:29.886 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T04:07:29.951 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-08T04:07:29.951 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T04:07:29.976 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T04:07:30.040 INFO:teuthology.run_tasks:Running task ceph... 2024-05-08T04:07:30.132 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-08T04:07:30.132 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 777 /var/log/ceph 2024-05-08T04:07:30.135 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 777 /var/log/ceph 2024-05-08T04:07:30.159 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-08T04:07:30.159 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-08T04:07:30.201 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-08T04:07:30.225 INFO:tasks.ceph:Creating extra log directories... 2024-05-08T04:07:30.226 DEBUG:teuthology.orchestra.run.smithi170:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-08T04:07:30.267 DEBUG:teuthology.orchestra.run.smithi187:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-08T04:07:30.303 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-08T04:07:30.303 INFO:tasks.ceph:config {'conf': {'client': {'debug cephsqlite': 20, 'debug client': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-08T04:07:30.303 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/kchai-2024-05-08_00:43:58-rados-wip-kefu-testing-2024-05-06-distro-default-smithi/7696821', 'branch': 'wip-kefu-testing-2024-05-06', 'description': 'rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} tasks/libcephsqlite}', 'email': None, 'first_in_suite': False, 'job_id': '7696821', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'kchai-2024-05-08_00:43:58-rados-wip-kefu-testing-2024-05-06-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-kefu-testing-2024-05-06'}, 'ceph': {'conf': {'client': {'debug cephsqlite': 20, 'debug client': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled'], 'sha1': '5ccf1c30cf724d5371f55439b3b3a90cb994fae2'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '5ccf1c30cf724d5371f55439b3b3a90cb994fae2'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-kefu-testing-2024-05-06', 'sha1': '5ccf1c30cf724d5371f55439b3b3a90cb994fae2'}}, 'owner': 'scheduled_kchai@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 2477, 'sha1': '5ccf1c30cf724d5371f55439b3b3a90cb994fae2', 'sleep_before_teardown': 0, 'subset': '1030/99999', 'suite': 'rados', 'suite_branch': 'wip-kefu-testing-2024-05-06', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_5ccf1c30cf724d5371f55439b3b3a90cb994fae2/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '5ccf1c30cf724d5371f55439b3b3a90cb994fae2', 'targets': {'smithi170.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDISdccAMpjTTpR17SreXiq5NUAYaHMBL61dw0VrbGVPmisU3BYRNBac0cDFl5tw2vHmlDZ77nLl7Dvsv2O1tW8=', 'smithi187.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA9FW+CA8Pfrhp6UvcPN2OnV1xZvW3bofLOIQf+CZelvy60JgzQQ8NVquLstj4GVp5uSLomwvG5Qy1KSQFa3+ig='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_system_packages': {'deb': ['sqlite3'], 'rpm': ['sqlite-devel']}, 'flavor': 'default', 'sha1': '5ccf1c30cf724d5371f55439b3b3a90cb994fae2'}}, {'ceph': None}, {'exec': {'client.0': ['ceph osd pool create cephsqlite', "ceph auth get-or-create client.libcephsqlite mon 'profile simple-rados-client-with-blocklist' osd 'allow rwx pool=cephsqlite' >> /etc/ceph/ceph.keyring"]}}, {'exec': {'client.0': ['ceph_test_libcephsqlite --id libcephsqlite --no-log-to-stderr']}}, {'workunit': {'clients': {'client.0': ['rados/test_libcephsqlite.sh cephsqlite']}, 'env': {'CEPH_ARGS': '--id libcephsqlite --no-log-to-stderr'}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '1ae7ad82388e92a475afff437d49054826c019a1', 'timestamp': '2024-05-08_00:43:58', 'tube': 'smithi', 'user': 'kchai', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-05-08T04:07:30.304 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-08T04:07:30.333 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-08T04:07:30.349 DEBUG:teuthology.orchestra.run.smithi170:> sudo install -d -m0777 -- /var/run/ceph 2024-05-08T04:07:30.389 DEBUG:teuthology.orchestra.run.smithi187:> sudo install -d -m0777 -- /var/run/ceph 2024-05-08T04:07:30.413 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-08T04:07:30.413 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T04:07:30.467 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T04:07:30.468 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_1 2024-05-08T04:07:30.523 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T04:07:30.523 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T04:07:30.523 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-08T04:07:30.523 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T04:07:30.523 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T04:07:30.523 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-05-08 04:07:17.362999077 +0000 2024-05-08T04:07:30.523 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-05-08 04:05:17.024225220 +0000 2024-05-08T04:07:30.524 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-05-08 04:05:17.024225220 +0000 2024-05-08T04:07:30.524 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-05-08 04:05:17.024225220 +0000 2024-05-08T04:07:30.524 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T04:07:30.587 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-05-08T04:07:30.587 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-05-08T04:07:30.587 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000106418 s, 4.8 MB/s 2024-05-08T04:07:30.588 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T04:07:30.645 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_2 2024-05-08T04:07:30.701 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T04:07:30.701 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T04:07:30.701 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-08T04:07:30.701 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T04:07:30.701 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T04:07:30.701 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-05-08 04:07:17.363999054 +0000 2024-05-08T04:07:30.701 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-05-08 04:05:17.353217584 +0000 2024-05-08T04:07:30.701 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-05-08 04:05:17.353217584 +0000 2024-05-08T04:07:30.701 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-05-08 04:05:17.353217584 +0000 2024-05-08T04:07:30.702 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T04:07:30.764 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-05-08T04:07:30.764 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-05-08T04:07:30.764 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000100104 s, 5.1 MB/s 2024-05-08T04:07:30.765 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T04:07:30.822 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_3 2024-05-08T04:07:30.878 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T04:07:30.878 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T04:07:30.878 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-08T04:07:30.878 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T04:07:30.878 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T04:07:30.878 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-05-08 04:07:17.363999054 +0000 2024-05-08T04:07:30.878 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-05-08 04:05:17.694209670 +0000 2024-05-08T04:07:30.878 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-05-08 04:05:17.694209670 +0000 2024-05-08T04:07:30.878 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-05-08 04:05:17.694209670 +0000 2024-05-08T04:07:30.879 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T04:07:30.941 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-05-08T04:07:30.941 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-05-08T04:07:30.941 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 8.1988e-05 s, 6.2 MB/s 2024-05-08T04:07:30.942 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T04:07:30.997 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_4 2024-05-08T04:07:31.052 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T04:07:31.053 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T04:07:31.053 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-08T04:07:31.053 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T04:07:31.053 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T04:07:31.053 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-05-08 04:07:17.364999031 +0000 2024-05-08T04:07:31.053 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-05-08 04:05:18.030201871 +0000 2024-05-08T04:07:31.053 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-05-08 04:05:18.030201871 +0000 2024-05-08T04:07:31.053 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-05-08 04:05:18.030201871 +0000 2024-05-08T04:07:31.053 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T04:07:31.116 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-05-08T04:07:31.116 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-05-08T04:07:31.116 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 8.0995e-05 s, 6.3 MB/s 2024-05-08T04:07:31.117 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T04:07:31.175 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T04:07:31.175 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-08T04:07:31.175 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T04:07:31.193 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T04:07:31.193 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_1 2024-05-08T04:07:31.250 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T04:07:31.250 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T04:07:31.250 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-05-08T04:07:31.250 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T04:07:31.250 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T04:07:31.250 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-05-08 04:07:21.496846207 +0000 2024-05-08T04:07:31.250 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-05-08 04:04:11.875993389 +0000 2024-05-08T04:07:31.250 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-05-08 04:04:11.875993389 +0000 2024-05-08T04:07:31.250 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-05-08 04:04:11.875993389 +0000 2024-05-08T04:07:31.251 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T04:07:31.314 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-05-08T04:07:31.315 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-05-08T04:07:31.315 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000212364 s, 2.4 MB/s 2024-05-08T04:07:31.316 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T04:07:31.373 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_2 2024-05-08T04:07:31.431 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T04:07:31.431 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T04:07:31.431 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-08T04:07:31.431 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T04:07:31.431 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T04:07:31.431 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-05-08 04:07:21.496846207 +0000 2024-05-08T04:07:31.431 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-05-08 04:04:12.213988104 +0000 2024-05-08T04:07:31.431 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-05-08 04:04:12.213988104 +0000 2024-05-08T04:07:31.431 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-05-08 04:04:12.213988104 +0000 2024-05-08T04:07:31.432 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T04:07:31.496 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-05-08T04:07:31.496 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-05-08T04:07:31.496 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000249897 s, 2.0 MB/s 2024-05-08T04:07:31.497 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T04:07:31.555 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_3 2024-05-08T04:07:31.612 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T04:07:31.612 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T04:07:31.612 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-05-08T04:07:31.612 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T04:07:31.612 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T04:07:31.612 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-05-08 04:07:21.497846192 +0000 2024-05-08T04:07:31.612 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-05-08 04:04:12.576982429 +0000 2024-05-08T04:07:31.612 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-05-08 04:04:12.576982429 +0000 2024-05-08T04:07:31.612 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-05-08 04:04:12.576982429 +0000 2024-05-08T04:07:31.613 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T04:07:31.676 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-05-08T04:07:31.676 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-05-08T04:07:31.676 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000319295 s, 1.6 MB/s 2024-05-08T04:07:31.678 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T04:07:31.734 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_4 2024-05-08T04:07:31.790 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T04:07:31.791 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T04:07:31.791 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-05-08T04:07:31.791 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T04:07:31.791 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T04:07:31.791 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-05-08 04:07:21.497846192 +0000 2024-05-08T04:07:31.791 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-05-08 04:04:12.903977316 +0000 2024-05-08T04:07:31.791 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-05-08 04:04:12.903977316 +0000 2024-05-08T04:07:31.791 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: 2024-05-08 04:04:12.903977316 +0000 2024-05-08T04:07:31.791 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T04:07:31.854 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-05-08T04:07:31.854 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-05-08T04:07:31.854 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000188222 s, 2.7 MB/s 2024-05-08T04:07:31.856 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T04:07:31.912 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T04:07:31.912 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi170.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi187.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-08T04:07:31.912 INFO:tasks.ceph:Generating config... 2024-05-08T04:07:31.914 INFO:tasks.ceph:[client] debug cephsqlite = 20 2024-05-08T04:07:31.914 INFO:tasks.ceph:[client] debug client = 20 2024-05-08T04:07:31.914 INFO:tasks.ceph:[client] debug ms = 1 2024-05-08T04:07:31.914 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-05-08T04:07:31.914 INFO:tasks.ceph:[global] mon election default strategy = 3 2024-05-08T04:07:31.914 INFO:tasks.ceph:[global] ms bind msgr1 = False 2024-05-08T04:07:31.914 INFO:tasks.ceph:[global] ms bind msgr2 = True 2024-05-08T04:07:31.914 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-05-08T04:07:31.914 INFO:tasks.ceph:[global] ms type = async 2024-05-08T04:07:31.914 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-08T04:07:31.914 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-08T04:07:31.914 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-08T04:07:31.915 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-08T04:07:31.915 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-08T04:07:31.915 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2024-05-08T04:07:31.915 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-05-08T04:07:31.915 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-05-08T04:07:31.915 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-05-08T04:07:31.915 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-05-08T04:07:31.915 INFO:tasks.ceph:[mon] paxos service trim min = 10 2024-05-08T04:07:31.915 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-08T04:07:31.915 INFO:tasks.ceph:[osd] bluestore compression algorithm = zstd 2024-05-08T04:07:31.915 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-05-08T04:07:31.916 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-08T04:07:31.916 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-05-08T04:07:31.916 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-05-08T04:07:31.916 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-05-08T04:07:31.916 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-08T04:07:31.916 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-08T04:07:31.916 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-05-08T04:07:31.916 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-08T04:07:31.916 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-08T04:07:31.916 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-08T04:07:31.916 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-05-08T04:07:31.916 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-05-08T04:07:31.916 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-08T04:07:31.916 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-05-08T04:07:31.916 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-05-08T04:07:31.917 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-08T04:07:31.917 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-05-08T04:07:31.917 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-05-08T04:07:31.917 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-05-08T04:07:31.917 INFO:tasks.ceph:Setting up mon.a... 2024-05-08T04:07:31.917 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-08T04:07:31.965 INFO:teuthology.orchestra.run.smithi170.stdout:creating /etc/ceph/ceph.keyring 2024-05-08T04:07:31.968 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-08T04:07:32.011 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T04:07:32.076 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.170'), ('mon.c', '[v2:172.21.15.170:3301,v1:172.21.15.170:6790]'), ('mon.b', '172.21.15.187')] 2024-05-08T04:07:32.077 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.170,[v2:172.21.15.170:3301,v1:172.21.15.170:6790],172.21.15.187', 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug cephsqlite': 20, 'debug client': 20, 'debug ms': 1}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-05-08T04:07:32.078 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-08T04:07:32.078 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-08T04:07:32.132 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.170 --addv c '[v2:172.21.15.170:3301,v1:172.21.15.170:6790]' --add b 172.21.15.187 --print /home/ubuntu/cephtest/ceph.monmap 2024-05-08T04:07:32.219 INFO:teuthology.orchestra.run.smithi170.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-08T04:07:32.219 INFO:teuthology.orchestra.run.smithi170.stdout:monmaptool: generated fsid e5497bb3-afa8-4f24-8ef4-42911ffcc962 2024-05-08T04:07:32.219 INFO:teuthology.orchestra.run.smithi170.stdout:setting min_mon_release = quincy 2024-05-08T04:07:32.219 INFO:teuthology.orchestra.run.smithi170.stdout:epoch 0 2024-05-08T04:07:32.219 INFO:teuthology.orchestra.run.smithi170.stdout:fsid e5497bb3-afa8-4f24-8ef4-42911ffcc962 2024-05-08T04:07:32.219 INFO:teuthology.orchestra.run.smithi170.stdout:last_changed 2024-05-08T04:07:32.217785+0000 2024-05-08T04:07:32.219 INFO:teuthology.orchestra.run.smithi170.stdout:created 2024-05-08T04:07:32.217785+0000 2024-05-08T04:07:32.219 INFO:teuthology.orchestra.run.smithi170.stdout:min_mon_release 17 (quincy) 2024-05-08T04:07:32.219 INFO:teuthology.orchestra.run.smithi170.stdout:election_strategy: 3 2024-05-08T04:07:32.219 INFO:teuthology.orchestra.run.smithi170.stdout:0: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.a 2024-05-08T04:07:32.219 INFO:teuthology.orchestra.run.smithi170.stdout:1: [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] mon.b 2024-05-08T04:07:32.219 INFO:teuthology.orchestra.run.smithi170.stdout:2: [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] mon.c 2024-05-08T04:07:32.220 INFO:teuthology.orchestra.run.smithi170.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-08T04:07:32.221 DEBUG:teuthology.orchestra.run.smithi170:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-08T04:07:32.237 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID e5497bb3-afa8-4f24-8ef4-42911ffcc962... 2024-05-08T04:07:32.238 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T04:07:32.280 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T04:07:32.320 INFO:teuthology.orchestra.run.smithi170.stdout:[global] 2024-05-08T04:07:32.320 INFO:teuthology.orchestra.run.smithi170.stdout: chdir = "" 2024-05-08T04:07:32.320 INFO:teuthology.orchestra.run.smithi170.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T04:07:32.320 INFO:teuthology.orchestra.run.smithi170.stdout: auth supported = cephx 2024-05-08T04:07:32.320 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:32.321 INFO:teuthology.orchestra.run.smithi170.stdout: filestore xattr use omap = true 2024-05-08T04:07:32.321 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:32.321 INFO:teuthology.orchestra.run.smithi170.stdout: mon clock drift allowed = 1.000 2024-05-08T04:07:32.321 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:32.321 INFO:teuthology.orchestra.run.smithi170.stdout: osd crush chooseleaf type = 0 2024-05-08T04:07:32.321 INFO:teuthology.orchestra.run.smithi170.stdout: auth debug = true 2024-05-08T04:07:32.321 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:32.321 INFO:teuthology.orchestra.run.smithi170.stdout: ms die on old message = true 2024-05-08T04:07:32.321 INFO:teuthology.orchestra.run.smithi170.stdout: ms die on bug = true 2024-05-08T04:07:32.321 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:32.321 INFO:teuthology.orchestra.run.smithi170.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T04:07:32.321 INFO:teuthology.orchestra.run.smithi170.stdout: mon pg warn max object skew = 0 2024-05-08T04:07:32.321 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:32.321 INFO:teuthology.orchestra.run.smithi170.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T04:07:32.321 INFO:teuthology.orchestra.run.smithi170.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T04:07:32.322 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:32.322 INFO:teuthology.orchestra.run.smithi170.stdout: osd pool default size = 2 2024-05-08T04:07:32.322 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:32.322 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd allow primary affinity = true 2024-05-08T04:07:32.322 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd allow pg remap = true 2024-05-08T04:07:32.322 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on legacy crush tunables = false 2024-05-08T04:07:32.322 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on crush straw calc version zero = false 2024-05-08T04:07:32.322 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on no sortbitwise = false 2024-05-08T04:07:32.322 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on osd down out interval zero = false 2024-05-08T04:07:32.322 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on too few osds = false 2024-05-08T04:07:32.322 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T04:07:32.322 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T04:07:32.322 INFO:teuthology.orchestra.run.smithi170.stdout: mon_allow_pool_size_one = true 2024-05-08T04:07:32.322 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:32.322 INFO:teuthology.orchestra.run.smithi170.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T04:07:32.323 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:32.323 INFO:teuthology.orchestra.run.smithi170.stdout: osd default data pool replay window = 5 2024-05-08T04:07:32.323 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:32.323 INFO:teuthology.orchestra.run.smithi170.stdout: mon allow pool delete = true 2024-05-08T04:07:32.323 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:32.323 INFO:teuthology.orchestra.run.smithi170.stdout: mon cluster log file level = debug 2024-05-08T04:07:32.323 INFO:teuthology.orchestra.run.smithi170.stdout: debug asserts on shutdown = true 2024-05-08T04:07:32.323 INFO:teuthology.orchestra.run.smithi170.stdout: mon health detail to clog = false 2024-05-08T04:07:32.323 INFO:teuthology.orchestra.run.smithi170.stdout: mon host = "172.21.15.170,[v2:172.21.15.170:3301,v1:172.21.15.170:6790],172.21.15.187" 2024-05-08T04:07:32.323 INFO:teuthology.orchestra.run.smithi170.stdout: mon client directed command retry = 5 2024-05-08T04:07:32.323 INFO:teuthology.orchestra.run.smithi170.stdout: mon election default strategy = 3 2024-05-08T04:07:32.323 INFO:teuthology.orchestra.run.smithi170.stdout: ms bind msgr1 = False 2024-05-08T04:07:32.323 INFO:teuthology.orchestra.run.smithi170.stdout: ms bind msgr2 = True 2024-05-08T04:07:32.323 INFO:teuthology.orchestra.run.smithi170.stdout: ms inject socket failures = 5000 2024-05-08T04:07:32.323 INFO:teuthology.orchestra.run.smithi170.stdout: ms type = async 2024-05-08T04:07:32.324 INFO:teuthology.orchestra.run.smithi170.stdout: fsid = e5497bb3-afa8-4f24-8ef4-42911ffcc962 2024-05-08T04:07:32.324 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:32.324 INFO:teuthology.orchestra.run.smithi170.stdout:[osd] 2024-05-08T04:07:32.324 INFO:teuthology.orchestra.run.smithi170.stdout: osd journal size = 100 2024-05-08T04:07:32.324 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:32.324 INFO:teuthology.orchestra.run.smithi170.stdout: osd scrub load threshold = 5.0 2024-05-08T04:07:32.324 INFO:teuthology.orchestra.run.smithi170.stdout: osd scrub max interval = 600 2024-05-08T04:07:32.324 INFO:teuthology.orchestra.run.smithi170.stdout: osd mclock profile = high_recovery_ops 2024-05-08T04:07:32.324 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:32.324 INFO:teuthology.orchestra.run.smithi170.stdout: osd recover clone overlap = true 2024-05-08T04:07:32.324 INFO:teuthology.orchestra.run.smithi170.stdout: osd recovery max chunk = 1048576 2024-05-08T04:07:32.324 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:32.324 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug shutdown = true 2024-05-08T04:07:32.324 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug op order = true 2024-05-08T04:07:32.324 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug verify stray on activate = true 2024-05-08T04:07:32.325 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug trim objects = true 2024-05-08T04:07:32.325 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:32.325 INFO:teuthology.orchestra.run.smithi170.stdout: osd open classes on start = true 2024-05-08T04:07:32.325 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug pg log writeout = true 2024-05-08T04:07:32.325 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:32.325 INFO:teuthology.orchestra.run.smithi170.stdout: osd deep scrub update digest min age = 30 2024-05-08T04:07:32.325 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:32.325 INFO:teuthology.orchestra.run.smithi170.stdout: osd map max advance = 10 2024-05-08T04:07:32.325 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:32.325 INFO:teuthology.orchestra.run.smithi170.stdout: journal zero on create = true 2024-05-08T04:07:32.325 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:32.325 INFO:teuthology.orchestra.run.smithi170.stdout: filestore ondisk finisher threads = 3 2024-05-08T04:07:32.325 INFO:teuthology.orchestra.run.smithi170.stdout: filestore apply finisher threads = 3 2024-05-08T04:07:32.325 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:32.325 INFO:teuthology.orchestra.run.smithi170.stdout: bdev debug aio = true 2024-05-08T04:07:32.326 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug misdirected ops = true 2024-05-08T04:07:32.326 INFO:teuthology.orchestra.run.smithi170.stdout: bluestore block size = 96636764160 2024-05-08T04:07:32.326 INFO:teuthology.orchestra.run.smithi170.stdout: bluestore compression algorithm = zstd 2024-05-08T04:07:32.326 INFO:teuthology.orchestra.run.smithi170.stdout: bluestore compression mode = aggressive 2024-05-08T04:07:32.326 INFO:teuthology.orchestra.run.smithi170.stdout: bluestore fsck on mount = True 2024-05-08T04:07:32.326 INFO:teuthology.orchestra.run.smithi170.stdout: bluestore zero block detection = True 2024-05-08T04:07:32.326 INFO:teuthology.orchestra.run.smithi170.stdout: debug bluefs = 20 2024-05-08T04:07:32.326 INFO:teuthology.orchestra.run.smithi170.stdout: debug bluestore = 20 2024-05-08T04:07:32.326 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2024-05-08T04:07:32.326 INFO:teuthology.orchestra.run.smithi170.stdout: debug osd = 20 2024-05-08T04:07:32.326 INFO:teuthology.orchestra.run.smithi170.stdout: debug rocksdb = 10 2024-05-08T04:07:32.326 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T04:07:32.326 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd full ratio = 0.9 2024-05-08T04:07:32.326 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd nearfull ratio = 0.8 2024-05-08T04:07:32.326 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug verify cached snaps = True 2024-05-08T04:07:32.327 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug verify missing on start = True 2024-05-08T04:07:32.327 INFO:teuthology.orchestra.run.smithi170.stdout: osd failsafe full ratio = 0.95 2024-05-08T04:07:32.327 INFO:teuthology.orchestra.run.smithi170.stdout: osd mclock override recovery settings = True 2024-05-08T04:07:32.327 INFO:teuthology.orchestra.run.smithi170.stdout: osd objectstore = bluestore 2024-05-08T04:07:32.327 INFO:teuthology.orchestra.run.smithi170.stdout: osd op queue = debug_random 2024-05-08T04:07:32.327 INFO:teuthology.orchestra.run.smithi170.stdout: osd op queue cut off = debug_random 2024-05-08T04:07:32.327 INFO:teuthology.orchestra.run.smithi170.stdout: osd shutdown pgref assert = True 2024-05-08T04:07:32.327 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:32.327 INFO:teuthology.orchestra.run.smithi170.stdout:[mgr] 2024-05-08T04:07:32.327 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2024-05-08T04:07:32.327 INFO:teuthology.orchestra.run.smithi170.stdout: debug mgr = 20 2024-05-08T04:07:32.327 INFO:teuthology.orchestra.run.smithi170.stdout: debug mon = 20 2024-05-08T04:07:32.327 INFO:teuthology.orchestra.run.smithi170.stdout: debug auth = 20 2024-05-08T04:07:32.327 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min pgs per osd = 4 2024-05-08T04:07:32.327 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min bytes per osd = 10 2024-05-08T04:07:32.328 INFO:teuthology.orchestra.run.smithi170.stdout: mgr/telemetry/nag = false 2024-05-08T04:07:32.328 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:32.328 INFO:teuthology.orchestra.run.smithi170.stdout:[mon] 2024-05-08T04:07:32.328 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2024-05-08T04:07:32.328 INFO:teuthology.orchestra.run.smithi170.stdout: debug mon = 20 2024-05-08T04:07:32.328 INFO:teuthology.orchestra.run.smithi170.stdout: debug paxos = 20 2024-05-08T04:07:32.328 INFO:teuthology.orchestra.run.smithi170.stdout: debug auth = 20 2024-05-08T04:07:32.328 INFO:teuthology.orchestra.run.smithi170.stdout: mon data avail warn = 5 2024-05-08T04:07:32.328 INFO:teuthology.orchestra.run.smithi170.stdout: mon mgr mkfs grace = 240 2024-05-08T04:07:32.328 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min pgs per osd = 4 2024-05-08T04:07:32.328 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd reporter subtree level = osd 2024-05-08T04:07:32.328 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd prime pg temp = true 2024-05-08T04:07:32.328 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min bytes per osd = 10 2024-05-08T04:07:32.328 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:32.328 INFO:teuthology.orchestra.run.smithi170.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T04:07:32.329 INFO:teuthology.orchestra.run.smithi170.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T04:07:32.329 INFO:teuthology.orchestra.run.smithi170.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T04:07:32.329 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:32.329 INFO:teuthology.orchestra.run.smithi170.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T04:07:32.329 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T04:07:32.329 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T04:07:32.329 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:32.329 INFO:teuthology.orchestra.run.smithi170.stdout: # 1m isn't quite enough 2024-05-08T04:07:32.329 INFO:teuthology.orchestra.run.smithi170.stdout: mon_down_mkfs_grace = 2m 2024-05-08T04:07:32.329 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:32.329 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_filestore_osds = false 2024-05-08T04:07:32.329 INFO:teuthology.orchestra.run.smithi170.stdout: mon min osdmap epochs = 50 2024-05-08T04:07:32.329 INFO:teuthology.orchestra.run.smithi170.stdout: mon osdmap full prune interval = 2 2024-05-08T04:07:32.329 INFO:teuthology.orchestra.run.smithi170.stdout: mon osdmap full prune min = 15 2024-05-08T04:07:32.329 INFO:teuthology.orchestra.run.smithi170.stdout: mon osdmap full prune txsize = 2 2024-05-08T04:07:32.329 INFO:teuthology.orchestra.run.smithi170.stdout: mon scrub interval = 300 2024-05-08T04:07:32.330 INFO:teuthology.orchestra.run.smithi170.stdout: paxos service trim min = 10 2024-05-08T04:07:32.330 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:07:32.330 INFO:teuthology.orchestra.run.smithi170.stdout:[client] 2024-05-08T04:07:32.330 INFO:teuthology.orchestra.run.smithi170.stdout: rgw cache enabled = true 2024-05-08T04:07:32.330 INFO:teuthology.orchestra.run.smithi170.stdout: rgw enable ops log = true 2024-05-08T04:07:32.330 INFO:teuthology.orchestra.run.smithi170.stdout: rgw enable usage log = true 2024-05-08T04:07:32.330 INFO:teuthology.orchestra.run.smithi170.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T04:07:32.330 INFO:teuthology.orchestra.run.smithi170.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T04:07:32.330 INFO:teuthology.orchestra.run.smithi170.stdout: debug cephsqlite = 20 2024-05-08T04:07:32.330 INFO:teuthology.orchestra.run.smithi170.stdout: debug client = 20 2024-05-08T04:07:32.330 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2024-05-08T04:07:32.330 INFO:teuthology.orchestra.run.smithi170.stdout:[mon.a] 2024-05-08T04:07:32.330 INFO:teuthology.orchestra.run.smithi170.stdout:[mon.c] 2024-05-08T04:07:32.330 INFO:teuthology.orchestra.run.smithi170.stdout:[mon.b] 2024-05-08T04:07:32.331 INFO:teuthology.orchestra.run.smithi187.stdout:[global] 2024-05-08T04:07:32.331 INFO:teuthology.orchestra.run.smithi187.stdout: chdir = "" 2024-05-08T04:07:32.331 INFO:teuthology.orchestra.run.smithi187.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T04:07:32.331 INFO:teuthology.orchestra.run.smithi187.stdout: auth supported = cephx 2024-05-08T04:07:32.331 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:32.331 INFO:teuthology.orchestra.run.smithi187.stdout: filestore xattr use omap = true 2024-05-08T04:07:32.331 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:32.332 INFO:teuthology.orchestra.run.smithi187.stdout: mon clock drift allowed = 1.000 2024-05-08T04:07:32.332 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:32.332 INFO:teuthology.orchestra.run.smithi187.stdout: osd crush chooseleaf type = 0 2024-05-08T04:07:32.332 INFO:teuthology.orchestra.run.smithi187.stdout: auth debug = true 2024-05-08T04:07:32.332 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:32.332 INFO:teuthology.orchestra.run.smithi187.stdout: ms die on old message = true 2024-05-08T04:07:32.332 INFO:teuthology.orchestra.run.smithi187.stdout: ms die on bug = true 2024-05-08T04:07:32.332 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:32.332 INFO:teuthology.orchestra.run.smithi187.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T04:07:32.332 INFO:teuthology.orchestra.run.smithi187.stdout: mon pg warn max object skew = 0 2024-05-08T04:07:32.332 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:32.332 INFO:teuthology.orchestra.run.smithi187.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T04:07:32.332 INFO:teuthology.orchestra.run.smithi187.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T04:07:32.332 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:32.332 INFO:teuthology.orchestra.run.smithi187.stdout: osd pool default size = 2 2024-05-08T04:07:32.333 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:32.333 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd allow primary affinity = true 2024-05-08T04:07:32.333 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd allow pg remap = true 2024-05-08T04:07:32.333 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on legacy crush tunables = false 2024-05-08T04:07:32.333 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on crush straw calc version zero = false 2024-05-08T04:07:32.333 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on no sortbitwise = false 2024-05-08T04:07:32.333 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on osd down out interval zero = false 2024-05-08T04:07:32.333 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on too few osds = false 2024-05-08T04:07:32.333 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T04:07:32.333 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T04:07:32.333 INFO:teuthology.orchestra.run.smithi187.stdout: mon_allow_pool_size_one = true 2024-05-08T04:07:32.333 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:32.333 INFO:teuthology.orchestra.run.smithi187.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T04:07:32.333 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:32.333 INFO:teuthology.orchestra.run.smithi187.stdout: osd default data pool replay window = 5 2024-05-08T04:07:32.333 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:32.334 INFO:teuthology.orchestra.run.smithi187.stdout: mon allow pool delete = true 2024-05-08T04:07:32.334 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:32.334 INFO:teuthology.orchestra.run.smithi187.stdout: mon cluster log file level = debug 2024-05-08T04:07:32.334 INFO:teuthology.orchestra.run.smithi187.stdout: debug asserts on shutdown = true 2024-05-08T04:07:32.334 INFO:teuthology.orchestra.run.smithi187.stdout: mon health detail to clog = false 2024-05-08T04:07:32.334 INFO:teuthology.orchestra.run.smithi187.stdout: mon host = "172.21.15.170,[v2:172.21.15.170:3301,v1:172.21.15.170:6790],172.21.15.187" 2024-05-08T04:07:32.334 INFO:teuthology.orchestra.run.smithi187.stdout: mon client directed command retry = 5 2024-05-08T04:07:32.334 INFO:teuthology.orchestra.run.smithi187.stdout: mon election default strategy = 3 2024-05-08T04:07:32.334 INFO:teuthology.orchestra.run.smithi187.stdout: ms bind msgr1 = False 2024-05-08T04:07:32.334 INFO:teuthology.orchestra.run.smithi187.stdout: ms bind msgr2 = True 2024-05-08T04:07:32.334 INFO:teuthology.orchestra.run.smithi187.stdout: ms inject socket failures = 5000 2024-05-08T04:07:32.334 INFO:teuthology.orchestra.run.smithi187.stdout: ms type = async 2024-05-08T04:07:32.334 INFO:teuthology.orchestra.run.smithi187.stdout: fsid = e5497bb3-afa8-4f24-8ef4-42911ffcc962 2024-05-08T04:07:32.334 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:32.335 INFO:teuthology.orchestra.run.smithi187.stdout:[osd] 2024-05-08T04:07:32.335 INFO:teuthology.orchestra.run.smithi187.stdout: osd journal size = 100 2024-05-08T04:07:32.335 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:32.335 INFO:teuthology.orchestra.run.smithi187.stdout: osd scrub load threshold = 5.0 2024-05-08T04:07:32.335 INFO:teuthology.orchestra.run.smithi187.stdout: osd scrub max interval = 600 2024-05-08T04:07:32.335 INFO:teuthology.orchestra.run.smithi187.stdout: osd mclock profile = high_recovery_ops 2024-05-08T04:07:32.335 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:32.335 INFO:teuthology.orchestra.run.smithi187.stdout: osd recover clone overlap = true 2024-05-08T04:07:32.335 INFO:teuthology.orchestra.run.smithi187.stdout: osd recovery max chunk = 1048576 2024-05-08T04:07:32.335 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:32.335 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug shutdown = true 2024-05-08T04:07:32.335 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug op order = true 2024-05-08T04:07:32.335 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug verify stray on activate = true 2024-05-08T04:07:32.335 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug trim objects = true 2024-05-08T04:07:32.335 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:32.336 INFO:teuthology.orchestra.run.smithi187.stdout: osd open classes on start = true 2024-05-08T04:07:32.336 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug pg log writeout = true 2024-05-08T04:07:32.336 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:32.336 INFO:teuthology.orchestra.run.smithi187.stdout: osd deep scrub update digest min age = 30 2024-05-08T04:07:32.336 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:32.336 INFO:teuthology.orchestra.run.smithi187.stdout: osd map max advance = 10 2024-05-08T04:07:32.336 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:32.336 INFO:teuthology.orchestra.run.smithi187.stdout: journal zero on create = true 2024-05-08T04:07:32.336 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:32.336 INFO:teuthology.orchestra.run.smithi187.stdout: filestore ondisk finisher threads = 3 2024-05-08T04:07:32.336 INFO:teuthology.orchestra.run.smithi187.stdout: filestore apply finisher threads = 3 2024-05-08T04:07:32.336 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:32.336 INFO:teuthology.orchestra.run.smithi187.stdout: bdev debug aio = true 2024-05-08T04:07:32.336 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug misdirected ops = true 2024-05-08T04:07:32.336 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore block size = 96636764160 2024-05-08T04:07:32.337 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore compression algorithm = zstd 2024-05-08T04:07:32.337 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore compression mode = aggressive 2024-05-08T04:07:32.337 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore fsck on mount = True 2024-05-08T04:07:32.337 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore zero block detection = True 2024-05-08T04:07:32.337 INFO:teuthology.orchestra.run.smithi187.stdout: debug bluefs = 20 2024-05-08T04:07:32.337 INFO:teuthology.orchestra.run.smithi187.stdout: debug bluestore = 20 2024-05-08T04:07:32.337 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-05-08T04:07:32.337 INFO:teuthology.orchestra.run.smithi187.stdout: debug osd = 20 2024-05-08T04:07:32.337 INFO:teuthology.orchestra.run.smithi187.stdout: debug rocksdb = 10 2024-05-08T04:07:32.337 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T04:07:32.337 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd full ratio = 0.9 2024-05-08T04:07:32.337 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd nearfull ratio = 0.8 2024-05-08T04:07:32.337 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug verify cached snaps = True 2024-05-08T04:07:32.337 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug verify missing on start = True 2024-05-08T04:07:32.337 INFO:teuthology.orchestra.run.smithi187.stdout: osd failsafe full ratio = 0.95 2024-05-08T04:07:32.338 INFO:teuthology.orchestra.run.smithi187.stdout: osd mclock override recovery settings = True 2024-05-08T04:07:32.338 INFO:teuthology.orchestra.run.smithi187.stdout: osd objectstore = bluestore 2024-05-08T04:07:32.338 INFO:teuthology.orchestra.run.smithi187.stdout: osd op queue = debug_random 2024-05-08T04:07:32.338 INFO:teuthology.orchestra.run.smithi187.stdout: osd op queue cut off = debug_random 2024-05-08T04:07:32.338 INFO:teuthology.orchestra.run.smithi187.stdout: osd shutdown pgref assert = True 2024-05-08T04:07:32.338 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:32.338 INFO:teuthology.orchestra.run.smithi187.stdout:[mgr] 2024-05-08T04:07:32.338 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-05-08T04:07:32.338 INFO:teuthology.orchestra.run.smithi187.stdout: debug mgr = 20 2024-05-08T04:07:32.338 INFO:teuthology.orchestra.run.smithi187.stdout: debug mon = 20 2024-05-08T04:07:32.338 INFO:teuthology.orchestra.run.smithi187.stdout: debug auth = 20 2024-05-08T04:07:32.338 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min pgs per osd = 4 2024-05-08T04:07:32.338 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min bytes per osd = 10 2024-05-08T04:07:32.338 INFO:teuthology.orchestra.run.smithi187.stdout: mgr/telemetry/nag = false 2024-05-08T04:07:32.338 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:32.339 INFO:teuthology.orchestra.run.smithi187.stdout:[mon] 2024-05-08T04:07:32.339 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-05-08T04:07:32.339 INFO:teuthology.orchestra.run.smithi187.stdout: debug mon = 20 2024-05-08T04:07:32.339 INFO:teuthology.orchestra.run.smithi187.stdout: debug paxos = 20 2024-05-08T04:07:32.339 INFO:teuthology.orchestra.run.smithi187.stdout: debug auth = 20 2024-05-08T04:07:32.339 INFO:teuthology.orchestra.run.smithi187.stdout: mon data avail warn = 5 2024-05-08T04:07:32.339 INFO:teuthology.orchestra.run.smithi187.stdout: mon mgr mkfs grace = 240 2024-05-08T04:07:32.339 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min pgs per osd = 4 2024-05-08T04:07:32.339 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd reporter subtree level = osd 2024-05-08T04:07:32.339 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd prime pg temp = true 2024-05-08T04:07:32.339 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min bytes per osd = 10 2024-05-08T04:07:32.339 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:32.339 INFO:teuthology.orchestra.run.smithi187.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T04:07:32.339 INFO:teuthology.orchestra.run.smithi187.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T04:07:32.339 INFO:teuthology.orchestra.run.smithi187.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T04:07:32.340 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:32.340 INFO:teuthology.orchestra.run.smithi187.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T04:07:32.340 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T04:07:32.340 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T04:07:32.340 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:32.340 INFO:teuthology.orchestra.run.smithi187.stdout: # 1m isn't quite enough 2024-05-08T04:07:32.340 INFO:teuthology.orchestra.run.smithi187.stdout: mon_down_mkfs_grace = 2m 2024-05-08T04:07:32.340 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:32.340 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_filestore_osds = false 2024-05-08T04:07:32.340 INFO:teuthology.orchestra.run.smithi187.stdout: mon min osdmap epochs = 50 2024-05-08T04:07:32.340 INFO:teuthology.orchestra.run.smithi187.stdout: mon osdmap full prune interval = 2 2024-05-08T04:07:32.340 INFO:teuthology.orchestra.run.smithi187.stdout: mon osdmap full prune min = 15 2024-05-08T04:07:32.340 INFO:teuthology.orchestra.run.smithi187.stdout: mon osdmap full prune txsize = 2 2024-05-08T04:07:32.340 INFO:teuthology.orchestra.run.smithi187.stdout: mon scrub interval = 300 2024-05-08T04:07:32.340 INFO:teuthology.orchestra.run.smithi187.stdout: paxos service trim min = 10 2024-05-08T04:07:32.340 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-05-08T04:07:32.341 INFO:teuthology.orchestra.run.smithi187.stdout:[client] 2024-05-08T04:07:32.341 INFO:teuthology.orchestra.run.smithi187.stdout: rgw cache enabled = true 2024-05-08T04:07:32.341 INFO:teuthology.orchestra.run.smithi187.stdout: rgw enable ops log = true 2024-05-08T04:07:32.341 INFO:teuthology.orchestra.run.smithi187.stdout: rgw enable usage log = true 2024-05-08T04:07:32.341 INFO:teuthology.orchestra.run.smithi187.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T04:07:32.341 INFO:teuthology.orchestra.run.smithi187.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T04:07:32.341 INFO:teuthology.orchestra.run.smithi187.stdout: debug cephsqlite = 20 2024-05-08T04:07:32.341 INFO:teuthology.orchestra.run.smithi187.stdout: debug client = 20 2024-05-08T04:07:32.341 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-05-08T04:07:32.341 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.a] 2024-05-08T04:07:32.341 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.c] 2024-05-08T04:07:32.341 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.b] 2024-05-08T04:07:32.343 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-08T04:07:32.343 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-08T04:07:32.437 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-08T04:07:32.438 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-08T04:07:32.438 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-08T04:07:32.453 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-08T04:07:32.453 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-08T04:07:32.509 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi170.front.sepia.ceph.com 2024-05-08T04:07:32.509 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-08T04:07:32.509 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-08T04:07:32.510 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T04:07:32.583 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-08T04:07:32.584 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-08T04:07:32.639 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi187.front.sepia.ceph.com 2024-05-08T04:07:32.639 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-08T04:07:32.639 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-08T04:07:32.639 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T04:07:32.674 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-08T04:07:32.674 DEBUG:teuthology.orchestra.run.smithi187:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-08T04:07:32.729 INFO:tasks.ceph:Setting up mon nodes... 2024-05-08T04:07:32.730 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-08T04:07:32.730 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-08T04:07:32.781 INFO:teuthology.orchestra.run.smithi170.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-08T04:07:32.784 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-08T04:07:32.839 INFO:teuthology.orchestra.run.smithi187.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-08T04:07:32.842 INFO:tasks.ceph:Setting up mds nodes... 2024-05-08T04:07:32.842 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-08T04:07:32.842 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-08T04:07:32.883 INFO:teuthology.orchestra.run.smithi170.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-08T04:07:32.896 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-08T04:07:32.936 INFO:teuthology.orchestra.run.smithi187.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-08T04:07:32.948 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-08T04:07:32.948 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi170.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi187.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-08T04:07:32.949 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-08T04:07:32.973 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T04:07:32.973 INFO:tasks.ceph:role: osd.0 2024-05-08T04:07:32.973 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi170.front.sepia.ceph.com 2024-05-08T04:07:32.974 DEBUG:teuthology.orchestra.run.smithi170:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-08T04:07:33.038 INFO:teuthology.orchestra.run.smithi170.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T04:07:33.038 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T04:07:33.038 INFO:teuthology.orchestra.run.smithi170.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T04:07:33.038 INFO:teuthology.orchestra.run.smithi170.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T04:07:33.038 INFO:teuthology.orchestra.run.smithi170.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T04:07:33.038 INFO:teuthology.orchestra.run.smithi170.stdout: = sunit=0 swidth=0 blks 2024-05-08T04:07:33.038 INFO:teuthology.orchestra.run.smithi170.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T04:07:33.038 INFO:teuthology.orchestra.run.smithi170.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T04:07:33.038 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T04:07:33.038 INFO:teuthology.orchestra.run.smithi170.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T04:07:33.070 INFO:teuthology.orchestra.run.smithi170.stdout:Discarding blocks...Done. 2024-05-08T04:07:33.072 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi170.front.sepia.ceph.com -o noatime 2024-05-08T04:07:33.073 DEBUG:teuthology.orchestra.run.smithi170:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-08T04:07:33.142 DEBUG:teuthology.orchestra.run.smithi170:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-08T04:07:33.213 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-08T04:07:33.278 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T04:07:33.278 INFO:tasks.ceph:role: osd.1 2024-05-08T04:07:33.278 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi170.front.sepia.ceph.com 2024-05-08T04:07:33.278 DEBUG:teuthology.orchestra.run.smithi170:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-08T04:07:33.342 INFO:teuthology.orchestra.run.smithi170.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T04:07:33.342 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T04:07:33.342 INFO:teuthology.orchestra.run.smithi170.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T04:07:33.342 INFO:teuthology.orchestra.run.smithi170.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T04:07:33.343 INFO:teuthology.orchestra.run.smithi170.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T04:07:33.343 INFO:teuthology.orchestra.run.smithi170.stdout: = sunit=0 swidth=0 blks 2024-05-08T04:07:33.343 INFO:teuthology.orchestra.run.smithi170.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T04:07:33.343 INFO:teuthology.orchestra.run.smithi170.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T04:07:33.343 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T04:07:33.343 INFO:teuthology.orchestra.run.smithi170.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T04:07:33.375 INFO:teuthology.orchestra.run.smithi170.stdout:Discarding blocks...Done. 2024-05-08T04:07:33.377 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi170.front.sepia.ceph.com -o noatime 2024-05-08T04:07:33.377 DEBUG:teuthology.orchestra.run.smithi170:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-08T04:07:33.451 DEBUG:teuthology.orchestra.run.smithi170:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-08T04:07:33.519 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-08T04:07:33.584 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T04:07:33.584 INFO:tasks.ceph:role: osd.2 2024-05-08T04:07:33.584 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi170.front.sepia.ceph.com 2024-05-08T04:07:33.584 DEBUG:teuthology.orchestra.run.smithi170:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-08T04:07:33.648 INFO:teuthology.orchestra.run.smithi170.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T04:07:33.648 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T04:07:33.648 INFO:teuthology.orchestra.run.smithi170.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T04:07:33.648 INFO:teuthology.orchestra.run.smithi170.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T04:07:33.648 INFO:teuthology.orchestra.run.smithi170.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T04:07:33.648 INFO:teuthology.orchestra.run.smithi170.stdout: = sunit=0 swidth=0 blks 2024-05-08T04:07:33.648 INFO:teuthology.orchestra.run.smithi170.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T04:07:33.648 INFO:teuthology.orchestra.run.smithi170.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T04:07:33.648 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T04:07:33.649 INFO:teuthology.orchestra.run.smithi170.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T04:07:33.680 INFO:teuthology.orchestra.run.smithi170.stdout:Discarding blocks...Done. 2024-05-08T04:07:33.682 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi170.front.sepia.ceph.com -o noatime 2024-05-08T04:07:33.682 DEBUG:teuthology.orchestra.run.smithi170:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-08T04:07:33.756 DEBUG:teuthology.orchestra.run.smithi170:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-08T04:07:33.824 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-08T04:07:33.888 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T04:07:33.888 INFO:tasks.ceph:role: osd.3 2024-05-08T04:07:33.888 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi170.front.sepia.ceph.com 2024-05-08T04:07:33.888 DEBUG:teuthology.orchestra.run.smithi170:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-08T04:07:33.952 INFO:teuthology.orchestra.run.smithi170.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T04:07:33.952 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T04:07:33.953 INFO:teuthology.orchestra.run.smithi170.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T04:07:33.953 INFO:teuthology.orchestra.run.smithi170.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T04:07:33.953 INFO:teuthology.orchestra.run.smithi170.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T04:07:33.953 INFO:teuthology.orchestra.run.smithi170.stdout: = sunit=0 swidth=0 blks 2024-05-08T04:07:33.953 INFO:teuthology.orchestra.run.smithi170.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T04:07:33.953 INFO:teuthology.orchestra.run.smithi170.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T04:07:33.953 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T04:07:33.953 INFO:teuthology.orchestra.run.smithi170.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T04:07:33.986 INFO:teuthology.orchestra.run.smithi170.stdout:Discarding blocks...Done. 2024-05-08T04:07:33.987 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi170.front.sepia.ceph.com -o noatime 2024-05-08T04:07:33.988 DEBUG:teuthology.orchestra.run.smithi170:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-08T04:07:34.057 DEBUG:teuthology.orchestra.run.smithi170:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-08T04:07:34.127 DEBUG:teuthology.orchestra.run.smithi170:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T04:07:34.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:07:34.256+0000 7f29c6011640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-08T04:07:34.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:07:34.257+0000 7f29c6011640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-08T04:07:34.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:07:34.257+0000 7f29c6011640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T04:07:34.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:07:34.257+0000 7f29c6011640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T04:07:34.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:07:34.257+0000 7f29c6011640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T04:07:34.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:07:34.257+0000 7f29c6011640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-08T04:07:37.109 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-08T04:07:37.138 DEBUG:teuthology.orchestra.run.smithi170:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T04:07:37.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:07:37.225+0000 7fe225d01640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-08T04:07:37.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:07:37.225+0000 7fe225d01640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-08T04:07:37.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:07:37.225+0000 7fe225d01640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T04:07:37.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:07:37.225+0000 7fe225d01640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T04:07:37.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:07:37.225+0000 7fe225d01640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T04:07:37.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:07:37.225+0000 7fe225d01640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-08T04:07:40.067 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-08T04:07:40.097 DEBUG:teuthology.orchestra.run.smithi170:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T04:07:40.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:07:40.183+0000 7fe6ca785640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-08T04:07:40.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:07:40.183+0000 7fe6ca785640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-08T04:07:40.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:07:40.183+0000 7fe6ca785640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T04:07:40.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:07:40.183+0000 7fe6ca785640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T04:07:40.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:07:40.183+0000 7fe6ca785640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T04:07:40.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:07:40.184+0000 7fe6ca785640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-08T04:07:43.035 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-08T04:07:43.064 DEBUG:teuthology.orchestra.run.smithi170:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T04:07:43.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:07:43.152+0000 7f5d71780640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-08T04:07:43.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:07:43.152+0000 7f5d71780640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-08T04:07:43.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:07:43.152+0000 7f5d71780640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T04:07:43.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:07:43.152+0000 7f5d71780640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T04:07:43.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:07:43.152+0000 7f5d71780640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T04:07:43.155 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:07:43.153+0000 7f5d71780640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-08T04:07:45.995 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-08T04:07:46.020 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-08T04:07:46.050 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T04:07:46.050 INFO:tasks.ceph:role: osd.4 2024-05-08T04:07:46.050 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi187.front.sepia.ceph.com 2024-05-08T04:07:46.050 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-08T04:07:46.125 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T04:07:46.125 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T04:07:46.125 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T04:07:46.125 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T04:07:46.125 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T04:07:46.125 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2024-05-08T04:07:46.125 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T04:07:46.125 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T04:07:46.126 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T04:07:46.126 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T04:07:46.608 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2024-05-08T04:07:46.610 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2024-05-08T04:07:46.610 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-08T04:07:46.735 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-08T04:07:46.769 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-08T04:07:46.835 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T04:07:46.835 INFO:tasks.ceph:role: osd.5 2024-05-08T04:07:46.835 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi187.front.sepia.ceph.com 2024-05-08T04:07:46.836 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-08T04:07:46.909 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T04:07:46.909 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T04:07:46.909 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T04:07:46.909 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T04:07:46.909 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T04:07:46.909 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2024-05-08T04:07:46.909 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T04:07:46.909 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T04:07:46.909 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T04:07:46.909 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T04:07:47.370 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2024-05-08T04:07:47.372 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2024-05-08T04:07:47.372 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-08T04:07:47.477 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-08T04:07:47.505 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-08T04:07:47.570 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T04:07:47.570 INFO:tasks.ceph:role: osd.6 2024-05-08T04:07:47.570 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi187.front.sepia.ceph.com 2024-05-08T04:07:47.570 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-08T04:07:47.643 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T04:07:47.643 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T04:07:47.643 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T04:07:47.643 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T04:07:47.643 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T04:07:47.643 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2024-05-08T04:07:47.643 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T04:07:47.643 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T04:07:47.643 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T04:07:47.643 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T04:07:48.134 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2024-05-08T04:07:48.136 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2024-05-08T04:07:48.136 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-08T04:07:48.288 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-08T04:07:48.318 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-08T04:07:48.382 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T04:07:48.383 INFO:tasks.ceph:role: osd.7 2024-05-08T04:07:48.383 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi187.front.sepia.ceph.com 2024-05-08T04:07:48.383 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-08T04:07:48.455 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T04:07:48.455 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T04:07:48.455 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T04:07:48.455 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T04:07:48.455 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T04:07:48.455 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2024-05-08T04:07:48.455 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T04:07:48.455 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T04:07:48.455 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T04:07:48.456 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T04:07:48.984 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2024-05-08T04:07:48.986 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2024-05-08T04:07:48.986 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-08T04:07:49.118 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-08T04:07:49.152 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T04:07:49.250 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:07:49.247+0000 7f3645e14640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-08T04:07:49.250 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:07:49.248+0000 7f3645e14640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-08T04:07:49.250 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:07:49.248+0000 7f3645e14640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T04:07:49.250 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:07:49.248+0000 7f3645e14640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T04:07:49.250 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:07:49.248+0000 7f3645e14640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T04:07:49.251 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:07:49.248+0000 7f3645e14640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-08T04:07:52.103 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-08T04:07:52.129 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T04:07:52.218 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:07:52.215+0000 7fb5c000c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-08T04:07:52.218 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:07:52.216+0000 7fb5c000c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-08T04:07:52.218 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:07:52.216+0000 7fb5c000c640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T04:07:52.218 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:07:52.216+0000 7fb5c000c640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T04:07:52.218 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:07:52.216+0000 7fb5c000c640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T04:07:52.218 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:07:52.216+0000 7fb5c000c640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-08T04:07:55.069 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-08T04:07:55.099 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T04:07:55.188 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:07:55.186+0000 7f91b2e21640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-08T04:07:55.188 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:07:55.186+0000 7f91b2e21640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-08T04:07:55.188 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:07:55.186+0000 7f91b2e21640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-08T04:07:55.188 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:07:55.186+0000 7f91b2e21640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-08T04:07:55.189 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:07:55.186+0000 7f91b2e21640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-08T04:07:55.189 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:07:55.187+0000 7f91b2e21640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-08T04:07:58.052 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-08T04:07:58.077 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T04:07:58.164 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:07:58.162+0000 7f652e01a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-08T04:07:58.164 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:07:58.162+0000 7f652e01a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-08T04:07:58.164 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:07:58.162+0000 7f652e01a640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-08T04:07:58.164 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:07:58.162+0000 7f652e01a640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-08T04:07:58.165 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:07:58.162+0000 7f652e01a640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-08T04:07:58.165 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:07:58.163+0000 7f652e01a640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-08T04:08:01.019 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-08T04:08:01.046 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-08T04:08:01.046 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-08T04:08:01.046 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-08T04:08:01.076 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-08T04:08:01.076 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-08T04:08:01.144 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-08T04:08:01.144 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-08T04:08:01.212 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-08T04:08:01.212 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-08T04:08:01.278 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-08T04:08:01.278 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-08T04:08:01.345 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-08T04:08:01.345 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-08T04:08:01.372 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-08T04:08:01.372 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-08T04:08:01.437 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-08T04:08:01.438 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-08T04:08:01.506 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-08T04:08:01.506 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-08T04:08:01.572 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-08T04:08:01.572 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-08T04:08:01.638 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-08T04:08:01.638 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-08T04:08:01.656 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-08T04:08:01.656 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-08T04:08:01.694 INFO:tasks.ceph:Adding keys to all mons... 2024-05-08T04:08:01.694 DEBUG:teuthology.orchestra.run.smithi170:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-08T04:08:01.698 DEBUG:teuthology.orchestra.run.smithi187:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-08T04:08:01.738 INFO:teuthology.orchestra.run.smithi170.stdout:[mgr.y] 2024-05-08T04:08:01.738 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCE+jpmS6qLLhAAm4ZzrdmzTBX5eIWFmUg63g== 2024-05-08T04:08:01.739 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.0] 2024-05-08T04:08:01.739 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCG+jpmBipfDxAA0ArtzB6Su61WaIne+kH3qA== 2024-05-08T04:08:01.739 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.1] 2024-05-08T04:08:01.739 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCJ+jpmwGh6DRAAwd1mkxpiAp7DhNl22N59uA== 2024-05-08T04:08:01.739 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.2] 2024-05-08T04:08:01.739 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCM+jpma4f9ChAAWI3tCz0woZpe+e9GguU2og== 2024-05-08T04:08:01.739 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.3] 2024-05-08T04:08:01.739 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCP+jpmq0ElCRAAU4fdgq62q/Vktm9o2ld8RQ== 2024-05-08T04:08:01.739 INFO:teuthology.orchestra.run.smithi170.stdout:[mgr.x] 2024-05-08T04:08:01.739 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCE+jpmUj4CMhAAzvQSRnDsIZle8A/OvbMvXA== 2024-05-08T04:08:01.739 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.4] 2024-05-08T04:08:01.739 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCV+jpmzKjWDhAAoZVI4Htynkk2hfg4lkjLRg== 2024-05-08T04:08:01.739 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.5] 2024-05-08T04:08:01.739 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCY+jpm8tbuDBAAQlxNOamfeCyXlwYjSyxRZw== 2024-05-08T04:08:01.739 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.6] 2024-05-08T04:08:01.740 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCb+jpmTVYpCxAA4XxXfH4fyh2Pb3oxhlLI8Q== 2024-05-08T04:08:01.740 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.7] 2024-05-08T04:08:01.740 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCe+jpmqaG7CRAAECDvEGjLxG221sENiTquNg== 2024-05-08T04:08:01.740 INFO:teuthology.orchestra.run.smithi170.stdout:[client.0] 2024-05-08T04:08:01.740 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCE+jpm4XieNBAA8QyszACYym+LzS3tEm/pkQ== 2024-05-08T04:08:01.740 INFO:teuthology.orchestra.run.smithi170.stdout:[client.1] 2024-05-08T04:08:01.740 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCE+jpmC5LDNxAAO25JK1wJ/9vVV1UoEUeNLQ== 2024-05-08T04:08:01.758 INFO:teuthology.orchestra.run.smithi187.stdout:[mgr.y] 2024-05-08T04:08:01.758 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCE+jpmS6qLLhAAm4ZzrdmzTBX5eIWFmUg63g== 2024-05-08T04:08:01.758 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.0] 2024-05-08T04:08:01.758 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCG+jpmBipfDxAA0ArtzB6Su61WaIne+kH3qA== 2024-05-08T04:08:01.758 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.1] 2024-05-08T04:08:01.758 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCJ+jpmwGh6DRAAwd1mkxpiAp7DhNl22N59uA== 2024-05-08T04:08:01.758 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.2] 2024-05-08T04:08:01.758 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCM+jpma4f9ChAAWI3tCz0woZpe+e9GguU2og== 2024-05-08T04:08:01.758 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.3] 2024-05-08T04:08:01.758 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCP+jpmq0ElCRAAU4fdgq62q/Vktm9o2ld8RQ== 2024-05-08T04:08:01.759 INFO:teuthology.orchestra.run.smithi187.stdout:[mgr.x] 2024-05-08T04:08:01.759 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCE+jpmUj4CMhAAzvQSRnDsIZle8A/OvbMvXA== 2024-05-08T04:08:01.759 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.4] 2024-05-08T04:08:01.759 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCV+jpmzKjWDhAAoZVI4Htynkk2hfg4lkjLRg== 2024-05-08T04:08:01.759 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.5] 2024-05-08T04:08:01.759 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCY+jpm8tbuDBAAQlxNOamfeCyXlwYjSyxRZw== 2024-05-08T04:08:01.759 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.6] 2024-05-08T04:08:01.759 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCb+jpmTVYpCxAA4XxXfH4fyh2Pb3oxhlLI8Q== 2024-05-08T04:08:01.759 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.7] 2024-05-08T04:08:01.759 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCe+jpmqaG7CRAAECDvEGjLxG221sENiTquNg== 2024-05-08T04:08:01.759 INFO:teuthology.orchestra.run.smithi187.stdout:[client.0] 2024-05-08T04:08:01.759 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCE+jpm4XieNBAA8QyszACYym+LzS3tEm/pkQ== 2024-05-08T04:08:01.759 INFO:teuthology.orchestra.run.smithi187.stdout:[client.1] 2024-05-08T04:08:01.759 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQCE+jpmC5LDNxAAO25JK1wJ/9vVV1UoEUeNLQ== 2024-05-08T04:08:01.760 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T04:08:01.783 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T04:08:01.851 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T04:08:01.853 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T04:08:01.904 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T04:08:01.907 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T04:08:01.957 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T04:08:01.959 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T04:08:02.010 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T04:08:02.012 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T04:08:02.063 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T04:08:02.065 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T04:08:02.115 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T04:08:02.117 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T04:08:02.167 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T04:08:02.170 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T04:08:02.220 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T04:08:02.222 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T04:08:02.273 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T04:08:02.275 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T04:08:02.325 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T04:08:02.330 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T04:08:02.383 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T04:08:02.386 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T04:08:02.438 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-08T04:08:02.439 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-08T04:08:02.463 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T04:08:02.559 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-08T04:08:02.583 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-08T04:08:02.648 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T04:08:02.746 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-08T04:08:02.772 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-08T04:08:02.796 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T04:08:02.891 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-08T04:08:02.916 DEBUG:teuthology.orchestra.run.smithi170:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-08T04:08:02.918 DEBUG:teuthology.orchestra.run.smithi187:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-08T04:08:02.971 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-08T04:08:02.972 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-08T04:08:02.972 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-08T04:08:02.974 INFO:tasks.ceph.mon.a:Started 2024-05-08T04:08:02.974 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-08T04:08:02.974 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-08T04:08:02.976 INFO:tasks.ceph.mon.c:Started 2024-05-08T04:08:02.976 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-08T04:08:02.976 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-08T04:08:03.014 INFO:tasks.ceph.mon.b:Started 2024-05-08T04:08:03.014 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-08T04:08:03.015 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-08T04:08:03.015 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-08T04:08:03.017 INFO:tasks.ceph.mgr.y:Started 2024-05-08T04:08:03.017 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-08T04:08:03.017 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-08T04:08:03.019 INFO:tasks.ceph.mgr.x:Started 2024-05-08T04:08:03.019 INFO:tasks.ceph:Setting crush tunables to default 2024-05-08T04:08:03.020 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd crush tunables default 2024-05-08T04:08:03.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:03.118+0000 7fdd8e633640 1 Processor -- start 2024-05-08T04:08:03.121 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:03.118+0000 7fdd8e633640 1 -- start start 2024-05-08T04:08:03.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:03.118+0000 7fdd8e633640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fdd880708c0 0x7fdd88141c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:03.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:03.118+0000 7fdd8e633640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd88073920 con 0x7fdd8813eeb0 2024-05-08T04:08:03.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:03.118+0000 7fdd8e633640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd8806ff70 con 0x7fdd88068c80 2024-05-08T04:08:03.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:03.118+0000 7fdd8e633640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7fdd88069060 con 0x7fdd880708c0 2024-05-08T04:08:03.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:03.118+0000 7fdd8cbd0640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fdd880708c0 0x7fdd88141c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:03.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:03.118+0000 7fdd8cbd0640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fdd880708c0 0x7fdd88141c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:54058/0 (socket says 172.21.15.170:54058) 2024-05-08T04:08:03.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:03.118+0000 7fdd877fe640 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fdd8813eeb0 0x7fdd88141560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:52282/0 (socket says 172.21.15.170:52282) 2024-05-08T04:08:03.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:03.118+0000 7fdd8cbd0640 1 -- 172.21.15.170:0/1033845701 learned_addr learned my addr 172.21.15.170:0/1033845701 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:03.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:03.118+0000 7fdd8cbd0640 1 -- 172.21.15.170:0/1033845701 >> v2:172.21.15.170:3300/0 conn(0x7fdd880708c0 msgr2=0x7fdd88141c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T04:08:03.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:03.118+0000 7fdd8cbd0640 1 -- 172.21.15.170:0/1033845701 >> v2:172.21.15.170:3300/0 conn(0x7fdd880708c0 msgr2=0x7fdd88141c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T04:08:03.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:03.118+0000 7fdd8cbd0640 1 --2- 172.21.15.170:0/1033845701 >> v2:172.21.15.170:3300/0 conn(0x7fdd880708c0 0x7fdd88141c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T04:08:03.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:03.119+0000 7fdd8cbd0640 1 --2- 172.21.15.170:0/1033845701 >> v2:172.21.15.170:3300/0 conn(0x7fdd880708c0 0x7fdd88141c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T04:08:03.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:03.320+0000 7fdd8cbd0640 1 --2- 172.21.15.170:0/1033845701 >> v2:172.21.15.170:3300/0 conn(0x7fdd880708c0 0x7fdd88141c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:03.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:03.320+0000 7fdd8cbd0640 1 -- 172.21.15.170:0/1033845701 >> v2:172.21.15.170:3300/0 conn(0x7fdd880708c0 msgr2=0x7fdd88141c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T04:08:03.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:03.320+0000 7fdd8cbd0640 1 -- 172.21.15.170:0/1033845701 >> v2:172.21.15.170:3300/0 conn(0x7fdd880708c0 msgr2=0x7fdd88141c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T04:08:03.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:03.320+0000 7fdd8cbd0640 1 --2- 172.21.15.170:0/1033845701 >> v2:172.21.15.170:3300/0 conn(0x7fdd880708c0 0x7fdd88141c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T04:08:03.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:03.320+0000 7fdd8cbd0640 1 --2- 172.21.15.170:0/1033845701 >> v2:172.21.15.170:3300/0 conn(0x7fdd880708c0 0x7fdd88141c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T04:08:03.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:03.721+0000 7fdd8cbd0640 1 --2- 172.21.15.170:0/1033845701 >> v2:172.21.15.170:3300/0 conn(0x7fdd880708c0 0x7fdd88141c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:03.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:03.721+0000 7fdd8cbd0640 1 -- 172.21.15.170:0/1033845701 >> v2:172.21.15.170:3300/0 conn(0x7fdd880708c0 msgr2=0x7fdd88141c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T04:08:03.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:03.721+0000 7fdd8cbd0640 1 -- 172.21.15.170:0/1033845701 >> v2:172.21.15.170:3300/0 conn(0x7fdd880708c0 msgr2=0x7fdd88141c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T04:08:03.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:03.721+0000 7fdd8cbd0640 1 --2- 172.21.15.170:0/1033845701 >> v2:172.21.15.170:3300/0 conn(0x7fdd880708c0 0x7fdd88141c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T04:08:03.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:03.721+0000 7fdd8cbd0640 1 --2- 172.21.15.170:0/1033845701 >> v2:172.21.15.170:3300/0 conn(0x7fdd880708c0 0x7fdd88141c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T04:08:04.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:04.523+0000 7fdd8cbd0640 1 --2- 172.21.15.170:0/1033845701 >> v2:172.21.15.170:3300/0 conn(0x7fdd880708c0 0x7fdd88141c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:04.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:04.523+0000 7fdd8cbd0640 1 -- 172.21.15.170:0/1033845701 >> v2:172.21.15.170:3300/0 conn(0x7fdd880708c0 msgr2=0x7fdd88141c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T04:08:04.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:04.523+0000 7fdd8cbd0640 1 -- 172.21.15.170:0/1033845701 >> v2:172.21.15.170:3300/0 conn(0x7fdd880708c0 msgr2=0x7fdd88141c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T04:08:04.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:04.523+0000 7fdd8cbd0640 1 --2- 172.21.15.170:0/1033845701 >> v2:172.21.15.170:3300/0 conn(0x7fdd880708c0 0x7fdd88141c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T04:08:04.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:04.523+0000 7fdd8cbd0640 1 --2- 172.21.15.170:0/1033845701 >> v2:172.21.15.170:3300/0 conn(0x7fdd880708c0 0x7fdd88141c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T04:08:06.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.119+0000 7fdd85ffb640 1 -- 172.21.15.170:0/1033845701 >> v2:172.21.15.170:3300/0 conn(0x7fdd880708c0 msgr2=0x7fdd88141c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T04:08:06.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.119+0000 7fdd85ffb640 1 --2- 172.21.15.170:0/1033845701 >> v2:172.21.15.170:3300/0 conn(0x7fdd880708c0 0x7fdd88141c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:06.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.119+0000 7fdd85ffb640 1 -- 172.21.15.170:0/1033845701 >> v1:172.21.15.187:6789/0 conn(0x7fdd88068c80 legacy=0x7fdd8813e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:06.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.119+0000 7fdd85ffb640 1 -- 172.21.15.170:0/1033845701 >> v1:172.21.15.170:6789/0 conn(0x7fdd8813eeb0 legacy=0x7fdd88141560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:06.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.119+0000 7fdd85ffb640 1 --2- 172.21.15.170:0/1033845701 >> v2:172.21.15.187:3300/0 conn(0x7fdd740036c0 0x7fdd74005b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:06.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.119+0000 7fdd85ffb640 1 --2- 172.21.15.170:0/1033845701 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd74006180 0x7fdd74008640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:06.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.120+0000 7fdd85ffb640 1 -- 172.21.15.170:0/1033845701 --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fdd88069060 con 0x7fdd740036c0 2024-05-08T04:08:06.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.120+0000 7fdd85ffb640 1 -- 172.21.15.170:0/1033845701 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fdd74008d70 con 0x7fdd74006180 2024-05-08T04:08:06.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.120+0000 7fdd877fe640 1 --2- 172.21.15.170:0/1033845701 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd74006180 0x7fdd74008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:06.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.120+0000 7fdd87fff640 1 --2- 172.21.15.170:0/1033845701 >> v2:172.21.15.187:3300/0 conn(0x7fdd740036c0 0x7fdd74005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:06.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.120+0000 7fdd877fe640 1 -- 172.21.15.170:0/1033845701 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd74006180 msgr2=0x7fdd74008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T04:08:06.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.120+0000 7fdd877fe640 1 -- 172.21.15.170:0/1033845701 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd74006180 msgr2=0x7fdd74008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T04:08:06.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.120+0000 7fdd877fe640 1 --2- 172.21.15.170:0/1033845701 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd74006180 0x7fdd74008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T04:08:06.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.120+0000 7fdd877fe640 1 --2- 172.21.15.170:0/1033845701 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd74006180 0x7fdd74008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T04:08:06.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.120+0000 7fdd87fff640 1 -- 172.21.15.170:0/1033845701 >> v2:172.21.15.187:3300/0 conn(0x7fdd740036c0 msgr2=0x7fdd74005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T04:08:06.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.120+0000 7fdd87fff640 1 -- 172.21.15.170:0/1033845701 >> v2:172.21.15.187:3300/0 conn(0x7fdd740036c0 msgr2=0x7fdd74005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T04:08:06.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.120+0000 7fdd87fff640 1 --2- 172.21.15.170:0/1033845701 >> v2:172.21.15.187:3300/0 conn(0x7fdd740036c0 0x7fdd74005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T04:08:06.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.120+0000 7fdd87fff640 1 --2- 172.21.15.170:0/1033845701 >> v2:172.21.15.187:3300/0 conn(0x7fdd740036c0 0x7fdd74005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T04:08:06.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.321+0000 7fdd877fe640 1 --2- 172.21.15.170:0/1033845701 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd74006180 0x7fdd74008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:06.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.321+0000 7fdd877fe640 1 -- 172.21.15.170:0/1033845701 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd74006180 msgr2=0x7fdd74008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T04:08:06.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.321+0000 7fdd877fe640 1 -- 172.21.15.170:0/1033845701 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd74006180 msgr2=0x7fdd74008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T04:08:06.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.321+0000 7fdd877fe640 1 --2- 172.21.15.170:0/1033845701 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd74006180 0x7fdd74008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T04:08:06.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.321+0000 7fdd877fe640 1 --2- 172.21.15.170:0/1033845701 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd74006180 0x7fdd74008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T04:08:06.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.321+0000 7fdd87fff640 1 --2- 172.21.15.170:0/1033845701 >> v2:172.21.15.187:3300/0 conn(0x7fdd740036c0 0x7fdd74005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:06.323 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.321+0000 7fdd87fff640 1 -- 172.21.15.170:0/1033845701 >> v2:172.21.15.187:3300/0 conn(0x7fdd740036c0 msgr2=0x7fdd74005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T04:08:06.324 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.321+0000 7fdd87fff640 1 -- 172.21.15.170:0/1033845701 >> v2:172.21.15.187:3300/0 conn(0x7fdd740036c0 msgr2=0x7fdd74005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T04:08:06.324 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.321+0000 7fdd87fff640 1 --2- 172.21.15.170:0/1033845701 >> v2:172.21.15.187:3300/0 conn(0x7fdd740036c0 0x7fdd74005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T04:08:06.324 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.321+0000 7fdd87fff640 1 --2- 172.21.15.170:0/1033845701 >> v2:172.21.15.187:3300/0 conn(0x7fdd740036c0 0x7fdd74005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T04:08:06.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.722+0000 7fdd877fe640 1 --2- 172.21.15.170:0/1033845701 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd74006180 0x7fdd74008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:06.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.722+0000 7fdd877fe640 1 -- 172.21.15.170:0/1033845701 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd74006180 msgr2=0x7fdd74008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T04:08:06.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.722+0000 7fdd877fe640 1 -- 172.21.15.170:0/1033845701 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd74006180 msgr2=0x7fdd74008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T04:08:06.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.722+0000 7fdd877fe640 1 --2- 172.21.15.170:0/1033845701 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd74006180 0x7fdd74008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T04:08:06.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.722+0000 7fdd877fe640 1 --2- 172.21.15.170:0/1033845701 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd74006180 0x7fdd74008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T04:08:06.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.722+0000 7fdd87fff640 1 --2- 172.21.15.170:0/1033845701 >> v2:172.21.15.187:3300/0 conn(0x7fdd740036c0 0x7fdd74005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:06.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.722+0000 7fdd87fff640 1 -- 172.21.15.170:0/1033845701 >> v2:172.21.15.187:3300/0 conn(0x7fdd740036c0 msgr2=0x7fdd74005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T04:08:06.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.722+0000 7fdd87fff640 1 -- 172.21.15.170:0/1033845701 >> v2:172.21.15.187:3300/0 conn(0x7fdd740036c0 msgr2=0x7fdd74005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T04:08:06.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.722+0000 7fdd87fff640 1 --2- 172.21.15.170:0/1033845701 >> v2:172.21.15.187:3300/0 conn(0x7fdd740036c0 0x7fdd74005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T04:08:06.725 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:06.722+0000 7fdd87fff640 1 --2- 172.21.15.170:0/1033845701 >> v2:172.21.15.187:3300/0 conn(0x7fdd740036c0 0x7fdd74005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T04:08:07.525 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:07.523+0000 7fdd877fe640 1 --2- 172.21.15.170:0/1033845701 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd74006180 0x7fdd74008640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:07.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:07.523+0000 7fdd877fe640 1 -- 172.21.15.170:0/1033845701 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd74006180 msgr2=0x7fdd74008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-05-08T04:08:07.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:07.523+0000 7fdd877fe640 1 -- 172.21.15.170:0/1033845701 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd74006180 msgr2=0x7fdd74008640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T04:08:07.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:07.523+0000 7fdd877fe640 1 --2- 172.21.15.170:0/1033845701 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd74006180 0x7fdd74008640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T04:08:07.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:07.523+0000 7fdd877fe640 1 --2- 172.21.15.170:0/1033845701 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd74006180 0x7fdd74008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T04:08:07.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:07.523+0000 7fdd87fff640 1 --2- 172.21.15.170:0/1033845701 >> v2:172.21.15.187:3300/0 conn(0x7fdd740036c0 0x7fdd74005b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:07.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:07.523+0000 7fdd87fff640 1 -- 172.21.15.170:0/1033845701 >> v2:172.21.15.187:3300/0 conn(0x7fdd740036c0 msgr2=0x7fdd74005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-08T04:08:07.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:07.523+0000 7fdd87fff640 1 -- 172.21.15.170:0/1033845701 >> v2:172.21.15.187:3300/0 conn(0x7fdd740036c0 msgr2=0x7fdd74005b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T04:08:07.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:07.523+0000 7fdd87fff640 1 --2- 172.21.15.170:0/1033845701 >> v2:172.21.15.187:3300/0 conn(0x7fdd740036c0 0x7fdd74005b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T04:08:07.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:07.523+0000 7fdd87fff640 1 --2- 172.21.15.170:0/1033845701 >> v2:172.21.15.187:3300/0 conn(0x7fdd740036c0 0x7fdd74005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T04:08:09.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.120+0000 7fdd85ffb640 1 -- 172.21.15.170:0/1033845701 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd74006180 msgr2=0x7fdd74008640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T04:08:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.120+0000 7fdd85ffb640 1 --2- 172.21.15.170:0/1033845701 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd74006180 0x7fdd74008640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.120+0000 7fdd85ffb640 1 -- 172.21.15.170:0/1033845701 >> v2:172.21.15.187:3300/0 conn(0x7fdd740036c0 msgr2=0x7fdd74005b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T04:08:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.120+0000 7fdd85ffb640 1 --2- 172.21.15.170:0/1033845701 >> v2:172.21.15.187:3300/0 conn(0x7fdd740036c0 0x7fdd74005b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.120+0000 7fdd8cbd0640 1 -- 172.21.15.170:0/1033845701 reap_dead start 2024-05-08T04:08:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.120+0000 7fdd85ffb640 1 --2- 172.21.15.170:0/1033845701 >> v2:172.21.15.170:3300/0 conn(0x7fdd7400c1d0 0x7fdd7400e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.120+0000 7fdd85ffb640 1 -- 172.21.15.170:0/1033845701 --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd880b2ba0 con 0x7fdd74007d50 2024-05-08T04:08:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.120+0000 7fdd85ffb640 1 -- 172.21.15.170:0/1033845701 --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd7400fb30 con 0x7fdd74003b30 2024-05-08T04:08:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.120+0000 7fdd85ffb640 1 -- 172.21.15.170:0/1033845701 --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7fdd88069060 con 0x7fdd7400c1d0 2024-05-08T04:08:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.120+0000 7fdd8cbd0640 1 --2- 172.21.15.170:0/1033845701 >> v2:172.21.15.170:3300/0 conn(0x7fdd7400c1d0 0x7fdd7400e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.120+0000 7fdd8cbd0640 1 -- 172.21.15.170:0/1033845701 >> v2:172.21.15.170:3300/0 conn(0x7fdd7400c1d0 msgr2=0x7fdd7400e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-08T04:08:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.120+0000 7fdd8cbd0640 1 -- 172.21.15.170:0/1033845701 >> v2:172.21.15.170:3300/0 conn(0x7fdd7400c1d0 msgr2=0x7fdd7400e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-08T04:08:09.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.120+0000 7fdd8cbd0640 1 --2- 172.21.15.170:0/1033845701 >> v2:172.21.15.170:3300/0 conn(0x7fdd7400c1d0 0x7fdd7400e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T04:08:09.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.120+0000 7fdd8cbd0640 1 --2- 172.21.15.170:0/1033845701 >> v2:172.21.15.170:3300/0 conn(0x7fdd7400c1d0 0x7fdd7400e5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T04:08:09.291 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.288+0000 7fdd86ffd640 1 -- 172.21.15.170:0/1033845701 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4239107132 0 0) 0x7fdd880b2ba0 con 0x7fdd74007d50 2024-05-08T04:08:09.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.288+0000 7fdd86ffd640 1 -- 172.21.15.170:0/1033845701 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd70003520 con 0x7fdd74007d50 2024-05-08T04:08:09.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.288+0000 7fdd86ffd640 1 -- 172.21.15.170:0/1033845701 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4257974859 0 0) 0x7fdd70003520 con 0x7fdd74007d50 2024-05-08T04:08:09.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.288+0000 7fdd86ffd640 1 -- 172.21.15.170:0/1033845701 >> v2:172.21.15.170:3300/0 conn(0x7fdd7400c1d0 msgr2=0x7fdd7400e5c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T04:08:09.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.288+0000 7fdd86ffd640 1 --2- 172.21.15.170:0/1033845701 >> v2:172.21.15.170:3300/0 conn(0x7fdd7400c1d0 0x7fdd7400e5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:09.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.288+0000 7fdd86ffd640 1 -- 172.21.15.170:0/1033845701 >> v1:172.21.15.187:6789/0 conn(0x7fdd74003b30 legacy=0x7fdd74006630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:09.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.289+0000 7fdd86ffd640 1 -- 172.21.15.170:0/1033845701 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd74010dd0 con 0x7fdd74007d50 2024-05-08T04:08:09.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.289+0000 7fdd86ffd640 1 -- 172.21.15.170:0/1033845701 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1210551688 0 0) 0x7fdd7c0014a0 con 0x7fdd74007d50 2024-05-08T04:08:09.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.289+0000 7fdd86ffd640 1 -- 172.21.15.170:0/1033845701 >> v1:172.21.15.170:6789/0 conn(0x7fdd74007d50 legacy=0x7fdd74008130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:09.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.289+0000 7fdd86ffd640 1 --2- 172.21.15.170:0/1033845701 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd700039b0 0x7fdd70003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:09.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.289+0000 7fdd86ffd640 1 --2- 172.21.15.170:0/1033845701 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd70004800 0x7fdd70004c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:09.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.289+0000 7fdd86ffd640 1 --2- 172.21.15.170:0/1033845701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd700052c0 0x7fdd7000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:09.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.289+0000 7fdd86ffd640 1 -- 172.21.15.170:0/1033845701 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdd88069060 con 0x7fdd700052c0 2024-05-08T04:08:09.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.289+0000 7fdd86ffd640 1 -- 172.21.15.170:0/1033845701 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7000c190 con 0x7fdd70004800 2024-05-08T04:08:09.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.289+0000 7fdd86ffd640 1 -- 172.21.15.170:0/1033845701 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fdd7000c390 con 0x7fdd700039b0 2024-05-08T04:08:09.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.289+0000 7fdd87fff640 1 --2- 172.21.15.170:0/1033845701 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd700039b0 0x7fdd70003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:09.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.289+0000 7fdd8cbd0640 1 --2- 172.21.15.170:0/1033845701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd700052c0 0x7fdd7000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:09.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.289+0000 7fdd877fe640 1 --2- 172.21.15.170:0/1033845701 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd70004800 0x7fdd70004c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:09.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.289+0000 7fdd8cbd0640 1 -- 172.21.15.170:0/1033845701 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd700039b0 msgr2=0x7fdd70003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:09.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.289+0000 7fdd8cbd0640 1 --2- 172.21.15.170:0/1033845701 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd700039b0 0x7fdd70003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:09.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.289+0000 7fdd8cbd0640 1 -- 172.21.15.170:0/1033845701 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd70004800 msgr2=0x7fdd70004c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:09.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.289+0000 7fdd8cbd0640 1 --2- 172.21.15.170:0/1033845701 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd70004800 0x7fdd70004c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:09.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.289+0000 7fdd8cbd0640 1 -- 172.21.15.170:0/1033845701 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd70003520 con 0x7fdd700052c0 2024-05-08T04:08:09.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.290+0000 7fdd8cbd0640 1 -- 172.21.15.170:0/1033845701 reap_dead start 2024-05-08T04:08:09.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.290+0000 7fdd8cbd0640 1 --2- 172.21.15.170:0/1033845701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd700052c0 0x7fdd7000b9c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdd8806ff00 tx=0x7fdd70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e340aca0c90bc322 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:09.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.290+0000 7fdd86ffd640 1 -- 172.21.15.170:0/1033845701 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd800097c0 con 0x7fdd700052c0 2024-05-08T04:08:09.294 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.290+0000 7fdd86ffd640 1 -- 172.21.15.170:0/1033845701 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd80003990 con 0x7fdd700052c0 2024-05-08T04:08:09.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.290+0000 7fdd86ffd640 1 -- 172.21.15.170:0/1033845701 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd80003d60 con 0x7fdd700052c0 2024-05-08T04:08:09.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.291+0000 7fdd8e633640 1 -- 172.21.15.170:0/1033845701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd700052c0 msgr2=0x7fdd7000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:09.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.291+0000 7fdd8e633640 1 --2- 172.21.15.170:0/1033845701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd700052c0 0x7fdd7000b9c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdd8806ff00 tx=0x7fdd70005830 comp rx=0 tx=0).stop 2024-05-08T04:08:09.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.291+0000 7fdd8e633640 1 -- 172.21.15.170:0/1033845701 shutdown_connections 2024-05-08T04:08:09.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.291+0000 7fdd8e633640 1 --2- 172.21.15.170:0/1033845701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd700052c0 0x7fdd7000b9c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:09.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.291+0000 7fdd8e633640 1 -- 172.21.15.170:0/1033845701 >> 172.21.15.170:0/1033845701 conn(0x7fdd880b5c20 msgr2=0x7fdd880b20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:09.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.291+0000 7fdd8e633640 1 -- 172.21.15.170:0/1033845701 shutdown_connections 2024-05-08T04:08:09.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.291+0000 7fdd8e633640 1 -- 172.21.15.170:0/1033845701 wait complete. 2024-05-08T04:08:09.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.293+0000 7fdd8e633640 1 Processor -- start 2024-05-08T04:08:09.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.294+0000 7fdd8e633640 1 -- start start 2024-05-08T04:08:09.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.294+0000 7fdd8e633640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd700052c0 0x7fdd880b1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:09.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.295+0000 7fdd8e633640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd88144270 0x7fdd88144ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:09.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.295+0000 7fdd8e633640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd881388d0 0x7fdd88138d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:09.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.295+0000 7fdd8e633640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8813eb30 con 0x7fdd881388d0 2024-05-08T04:08:09.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.295+0000 7fdd8e633640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8813e9b0 con 0x7fdd88144270 2024-05-08T04:08:09.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.295+0000 7fdd8e633640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fdd8813ecb0 con 0x7fdd700052c0 2024-05-08T04:08:09.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.295+0000 7fdd87fff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd700052c0 0x7fdd880b1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:09.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.295+0000 7fdd87fff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd700052c0 0x7fdd880b1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:60482/0 (socket says 172.21.15.170:60482) 2024-05-08T04:08:09.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.295+0000 7fdd87fff640 1 -- 172.21.15.170:0/3061410759 learned_addr learned my addr 172.21.15.170:0/3061410759 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:09.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.295+0000 7fdd877fe640 1 --2- 172.21.15.170:0/3061410759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd88144270 0x7fdd88144ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:09.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.295+0000 7fdd8cbd0640 1 --2- 172.21.15.170:0/3061410759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd881388d0 0x7fdd88138d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:09.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.295+0000 7fdd87fff640 1 -- 172.21.15.170:0/3061410759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd88144270 msgr2=0x7fdd88144ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:09.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.295+0000 7fdd87fff640 1 --2- 172.21.15.170:0/3061410759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd88144270 0x7fdd88144ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:09.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.295+0000 7fdd87fff640 1 -- 172.21.15.170:0/3061410759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd881388d0 msgr2=0x7fdd88138d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:09.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.295+0000 7fdd87fff640 1 --2- 172.21.15.170:0/3061410759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd881388d0 0x7fdd88138d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:09.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.295+0000 7fdd87fff640 1 -- 172.21.15.170:0/3061410759 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd881451e0 con 0x7fdd700052c0 2024-05-08T04:08:09.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.295+0000 7fdd87fff640 1 --2- 172.21.15.170:0/3061410759 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd700052c0 0x7fdd880b1980 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdd78007a80 tx=0x7fdd7800f8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:09.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.296+0000 7fdd84ff9640 1 -- 172.21.15.170:0/3061410759 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd780017c0 con 0x7fdd700052c0 2024-05-08T04:08:09.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.296+0000 7fdd84ff9640 1 -- 172.21.15.170:0/3061410759 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd78001960 con 0x7fdd700052c0 2024-05-08T04:08:09.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.296+0000 7fdd8e633640 1 -- 172.21.15.170:0/3061410759 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd88139290 con 0x7fdd700052c0 2024-05-08T04:08:09.299 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.296+0000 7fdd84ff9640 1 -- 172.21.15.170:0/3061410759 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd78001d10 con 0x7fdd700052c0 2024-05-08T04:08:09.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.296+0000 7fdd8e633640 1 -- 172.21.15.170:0/3061410759 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd881d4780 con 0x7fdd700052c0 2024-05-08T04:08:09.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.296+0000 7fdd84ff9640 1 -- 172.21.15.170:0/3061410759 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdd7801c410 con 0x7fdd700052c0 2024-05-08T04:08:09.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.296+0000 7fdd84ff9640 1 -- 172.21.15.170:0/3061410759 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdd78025900 con 0x7fdd700052c0 2024-05-08T04:08:09.300 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.297+0000 7fdd8e633640 1 -- 172.21.15.170:0/3061410759 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd881d5260 con 0x7fdd700052c0 2024-05-08T04:08:09.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.299+0000 7fdd84ff9640 1 -- 172.21.15.170:0/3061410759 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fdd78017050 con 0x7fdd700052c0 2024-05-08T04:08:09.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.366+0000 7fdd8e633640 1 -- 172.21.15.170:0/3061410759 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fdd88076a70 con 0x7fdd700052c0 2024-05-08T04:08:09.409 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-08T04:08:09.407+0000 7f80f120d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T04:08:09.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.492+0000 7fdd84ff9640 1 -- 172.21.15.170:0/3061410759 <== mon.2 v2:172.21.15.170:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fdd78025460 con 0x7fdd700052c0 2024-05-08T04:08:09.495 INFO:teuthology.orchestra.run.smithi170.stderr:adjusted tunables profile to default 2024-05-08T04:08:09.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.493+0000 7fdd8e633640 1 -- 172.21.15.170:0/3061410759 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd700052c0 msgr2=0x7fdd880b1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:09.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.493+0000 7fdd8e633640 1 --2- 172.21.15.170:0/3061410759 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd700052c0 0x7fdd880b1980 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdd78007a80 tx=0x7fdd7800f8b0 comp rx=0 tx=0).stop 2024-05-08T04:08:09.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.493+0000 7fdd8e633640 1 -- 172.21.15.170:0/3061410759 shutdown_connections 2024-05-08T04:08:09.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.493+0000 7fdd8e633640 1 --2- 172.21.15.170:0/3061410759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd88144270 0x7fdd88144ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:09.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.493+0000 7fdd8e633640 1 --2- 172.21.15.170:0/3061410759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd881388d0 0x7fdd88138d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:09.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.493+0000 7fdd8e633640 1 --2- 172.21.15.170:0/3061410759 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd700052c0 0x7fdd880b1980 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:09.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.493+0000 7fdd8e633640 1 -- 172.21.15.170:0/3061410759 >> 172.21.15.170:0/3061410759 conn(0x7fdd880b5c20 msgr2=0x7fdd88141ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:09.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.493+0000 7fdd8e633640 1 -- 172.21.15.170:0/3061410759 shutdown_connections 2024-05-08T04:08:09.496 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:09.494+0000 7fdd8e633640 1 -- 172.21.15.170:0/3061410759 wait complete. 2024-05-08T04:08:09.507 INFO:tasks.ceph:check_enable_crimson: False 2024-05-08T04:08:09.507 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-08T04:08:09.507 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-08T04:08:09.507 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-08T04:08:09.519 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-05-08T04:08:09.517+0000 7f7ca884b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T04:08:09.531 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-08T04:08:09.531 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-08T04:08:09.574 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-08T04:08:09.572+0000 7f80f120d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T04:08:09.598 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-08T04:08:09.598 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-08T04:08:09.627 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-08T04:08:09.625+0000 7f80f120d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T04:08:09.664 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-08T04:08:09.664 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-08T04:08:09.681 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-05-08T04:08:09.679+0000 7f7ca884b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T04:08:09.705 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-08T04:08:09.705 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-08T04:08:09.730 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-08T04:08:09.730 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-08T04:08:09.738 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-05-08T04:08:09.736+0000 7f7ca884b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T04:08:09.795 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-08T04:08:09.795 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-08T04:08:09.861 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-05-08T04:08:09.861 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-08T04:08:09.902 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-08T04:08:09.900+0000 7f80f120d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T04:08:09.926 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 6bca968b-d825-417a-b907-5312aad8dbcd 0 2024-05-08T04:08:09.950 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-08T04:08:09.948+0000 7f80f120d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T04:08:10.008 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-05-08T04:08:10.006+0000 7f7ca884b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T04:08:10.049 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.047+0000 7f7972e21640 1 Processor -- start 2024-05-08T04:08:10.049 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.047+0000 7f7972e21640 1 -- start start 2024-05-08T04:08:10.051 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.047+0000 7f7972e21640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f796c073fd0 0x7f796c071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:10.051 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.047+0000 7f7972e21640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f796c140440 0x7f796c0af600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:10.051 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.047+0000 7f7972e21640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f796c073280 con 0x7f796c0afb40 2024-05-08T04:08:10.051 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.047+0000 7f7972e21640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7f796c071a70 con 0x7f796c140440 2024-05-08T04:08:10.051 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.047+0000 7f7972e21640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f796c073050 con 0x7f796c073fd0 2024-05-08T04:08:10.051 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.048+0000 7f7970bbd640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f796c073fd0 0x7f796c071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:10.051 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.048+0000 7f796bfff640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f796c140440 0x7f796c0af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:10.051 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.048+0000 7f79713be640 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f796c0afb40 0x7f796c1439c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.187:53122/0 (socket says 172.21.15.187:53122) 2024-05-08T04:08:10.051 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.048+0000 7f79713be640 1 -- 172.21.15.187:0/3345849205 learned_addr learned my addr 172.21.15.187:0/3345849205 (peer_addr_for_me v1:172.21.15.187:0/0) 2024-05-08T04:08:10.051 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.048+0000 7f7970bbd640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f796c073fd0 0x7f796c071530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.187:37666/0 (socket says 172.21.15.187:37666) 2024-05-08T04:08:10.051 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.048+0000 7f7970bbd640 1 -- 172.21.15.187:0/3345849205 >> v2:172.21.15.170:3300/0 conn(0x7f796c140440 msgr2=0x7f796c0af600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:10.051 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.048+0000 7f7970bbd640 1 --2- 172.21.15.187:0/3345849205 >> v2:172.21.15.170:3300/0 conn(0x7f796c140440 0x7f796c0af600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.052 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.048+0000 7f7970bbd640 1 -- 172.21.15.187:0/3345849205 >> v1:172.21.15.170:6789/0 conn(0x7f796c0afb40 legacy=0x7f796c1439c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:10.052 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.048+0000 7f7970bbd640 1 -- 172.21.15.187:0/3345849205 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f796c073560 con 0x7f796c073fd0 2024-05-08T04:08:10.052 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.048+0000 7f796bfff640 1 --2- 172.21.15.187:0/3345849205 >> v2:172.21.15.170:3300/0 conn(0x7f796c140440 0x7f796c0af600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:08:10.052 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.049+0000 7f7970bbd640 1 --2- 172.21.15.187:0/3345849205 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f796c073fd0 0x7f796c071530 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f79600097a0 tx=0x7f796002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62ebf80458216ade server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:10.052 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.049+0000 7f796b7fe640 1 -- 172.21.15.187:0/3345849205 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7960035020 con 0x7f796c073fd0 2024-05-08T04:08:10.052 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.049+0000 7f796b7fe640 1 -- 172.21.15.187:0/3345849205 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7960031830 con 0x7f796c073fd0 2024-05-08T04:08:10.052 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.049+0000 7f796b7fe640 1 -- 172.21.15.187:0/3345849205 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7960031c00 con 0x7f796c073fd0 2024-05-08T04:08:10.054 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.049+0000 7f7972e21640 1 -- 172.21.15.187:0/3345849205 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f796c073fd0 msgr2=0x7f796c071530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:10.054 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.049+0000 7f7972e21640 1 --2- 172.21.15.187:0/3345849205 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f796c073fd0 0x7f796c071530 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f79600097a0 tx=0x7f796002b8d0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.054 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.049+0000 7f7972e21640 1 -- 172.21.15.187:0/3345849205 shutdown_connections 2024-05-08T04:08:10.055 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.049+0000 7f7972e21640 1 --2- 172.21.15.187:0/3345849205 >> v2:172.21.15.170:3300/0 conn(0x7f796c140440 0x7f796c0af600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.055 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.049+0000 7f7972e21640 1 --2- 172.21.15.187:0/3345849205 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f796c073fd0 0x7f796c071530 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.055 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.049+0000 7f7972e21640 1 -- 172.21.15.187:0/3345849205 >> 172.21.15.187:0/3345849205 conn(0x7f796c0b6020 msgr2=0x7f796c0b6430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:10.055 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.050+0000 7f7972e21640 1 -- 172.21.15.187:0/3345849205 shutdown_connections 2024-05-08T04:08:10.055 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.050+0000 7f7972e21640 1 -- 172.21.15.187:0/3345849205 wait complete. 2024-05-08T04:08:10.055 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.050+0000 7f7972e21640 1 Processor -- start 2024-05-08T04:08:10.055 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.051+0000 7f7972e21640 1 -- start start 2024-05-08T04:08:10.055 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.051+0000 7f7972e21640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f796c073fd0 0x7f796c13fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:10.055 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.051+0000 7f7972e21640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f796c0afb40 0x7f796c135210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:10.055 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.051+0000 7f7972e21640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f796c140440 0x7f796c135750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:10.055 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.051+0000 7f7972e21640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f796c0720b0 con 0x7f796c073fd0 2024-05-08T04:08:10.055 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.051+0000 7f7972e21640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f796c0710a0 con 0x7f796c140440 2024-05-08T04:08:10.055 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.051+0000 7f7972e21640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f796c0743b0 con 0x7f796c0afb40 2024-05-08T04:08:10.055 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.051+0000 7f796bfff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f796c0afb40 0x7f796c135210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:10.055 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.051+0000 7f79713be640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f796c140440 0x7f796c135750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:10.056 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.051+0000 7f7970bbd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f796c073fd0 0x7f796c13fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:10.056 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.051+0000 7f796bfff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f796c0afb40 0x7f796c135210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.187:37676/0 (socket says 172.21.15.187:37676) 2024-05-08T04:08:10.056 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.051+0000 7f796bfff640 1 -- 172.21.15.187:0/56669619 learned_addr learned my addr 172.21.15.187:0/56669619 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-05-08T04:08:10.056 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.052+0000 7f79713be640 1 -- 172.21.15.187:0/56669619 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f796c0afb40 msgr2=0x7f796c135210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:10.056 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.052+0000 7f79713be640 1 --2- 172.21.15.187:0/56669619 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f796c0afb40 0x7f796c135210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.056 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.052+0000 7f79713be640 1 -- 172.21.15.187:0/56669619 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f796c073fd0 msgr2=0x7f796c13fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:10.056 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.052+0000 7f79713be640 1 --2- 172.21.15.187:0/56669619 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f796c073fd0 0x7f796c13fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.056 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.052+0000 7f79713be640 1 -- 172.21.15.187:0/56669619 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f796c136730 con 0x7f796c140440 2024-05-08T04:08:10.056 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.052+0000 7f796bfff640 1 --2- 172.21.15.187:0/56669619 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f796c0afb40 0x7f796c135210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:08:10.056 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.052+0000 7f79713be640 1 --2- 172.21.15.187:0/56669619 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f796c140440 0x7f796c135750 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f796400cb10 tx=0x7f796400a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:10.056 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.052+0000 7f79697fa640 1 -- 172.21.15.187:0/56669619 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7964017070 con 0x7f796c140440 2024-05-08T04:08:10.056 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.052+0000 7f79697fa640 1 -- 172.21.15.187:0/56669619 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f796400ac70 con 0x7f796c140440 2024-05-08T04:08:10.056 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.052+0000 7f7972e21640 1 -- 172.21.15.187:0/56669619 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f796c136210 con 0x7f796c140440 2024-05-08T04:08:10.056 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.052+0000 7f79697fa640 1 -- 172.21.15.187:0/56669619 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7964012600 con 0x7f796c140440 2024-05-08T04:08:10.056 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.052+0000 7f7972e21640 1 -- 172.21.15.187:0/56669619 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f796c135e30 con 0x7f796c140440 2024-05-08T04:08:10.056 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.052+0000 7f79697fa640 1 -- 172.21.15.187:0/56669619 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f796401a070 con 0x7f796c140440 2024-05-08T04:08:10.057 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.052+0000 7f79697fa640 1 -- 172.21.15.187:0/56669619 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f796401cae0 con 0x7f796c140440 2024-05-08T04:08:10.057 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.053+0000 7f7972e21640 1 -- 172.21.15.187:0/56669619 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f796c13da70 con 0x7f796c140440 2024-05-08T04:08:10.057 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.055+0000 7f79697fa640 1 -- 172.21.15.187:0/56669619 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f796c13da70 con 0x7f796c140440 2024-05-08T04:08:10.058 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-05-08T04:08:10.056+0000 7f7ca884b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T04:08:10.127 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.125+0000 7f7972e21640 1 -- 172.21.15.187:0/56669619 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6bca968b-d825-417a-b907-5312aad8dbcd", "id": 0} v 0) -- 0x7f796c069bd0 con 0x7f796c140440 2024-05-08T04:08:10.133 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.131+0000 7f79697fa640 1 -- 172.21.15.187:0/56669619 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6bca968b-d825-417a-b907-5312aad8dbcd", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f79640127a0 con 0x7f796c140440 2024-05-08T04:08:10.133 INFO:teuthology.orchestra.run.smithi187.stdout:0 2024-05-08T04:08:10.134 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.132+0000 7f7972e21640 1 -- 172.21.15.187:0/56669619 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f796c140440 msgr2=0x7f796c135750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:10.134 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.132+0000 7f7972e21640 1 --2- 172.21.15.187:0/56669619 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f796c140440 0x7f796c135750 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f796400cb10 tx=0x7f796400a060 comp rx=0 tx=0).stop 2024-05-08T04:08:10.135 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.132+0000 7f7972e21640 1 -- 172.21.15.187:0/56669619 shutdown_connections 2024-05-08T04:08:10.135 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.132+0000 7f7972e21640 1 --2- 172.21.15.187:0/56669619 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f796c140440 0x7f796c135750 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.135 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.132+0000 7f7972e21640 1 --2- 172.21.15.187:0/56669619 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f796c0afb40 0x7f796c135210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.135 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.132+0000 7f7972e21640 1 --2- 172.21.15.187:0/56669619 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f796c073fd0 0x7f796c13fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.135 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.132+0000 7f7972e21640 1 -- 172.21.15.187:0/56669619 >> 172.21.15.187:0/56669619 conn(0x7f796c0b6020 msgr2=0x7f796c0b6400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:10.135 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.132+0000 7f7972e21640 1 -- 172.21.15.187:0/56669619 shutdown_connections 2024-05-08T04:08:10.135 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.132+0000 7f7972e21640 1 -- 172.21.15.187:0/56669619 wait complete. 2024-05-08T04:08:10.145 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 66600f89-4532-43b1-85d3-6d8c736cfaf2 1 2024-05-08T04:08:10.244 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.240+0000 7fbad1eb5640 1 Processor -- start 2024-05-08T04:08:10.244 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.241+0000 7fbad1eb5640 1 -- start start 2024-05-08T04:08:10.244 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.241+0000 7fbad1eb5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbacc075ea0 0x7fbacc13e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:10.244 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.241+0000 7fbad1eb5640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbacc0b0260 0x7fbacc13eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:10.244 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.241+0000 7fbad1eb5640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbacc0b24b0 con 0x7fbacc1415e0 2024-05-08T04:08:10.244 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.241+0000 7fbad1eb5640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fbacc076280 con 0x7fbacc0b0260 2024-05-08T04:08:10.244 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.241+0000 7fbad1eb5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fbacc0aa650 con 0x7fbacc075ea0 2024-05-08T04:08:10.244 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.241+0000 7fbac3fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbacc0b0260 0x7fbacc13eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:10.244 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.241+0000 7fbac3fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbacc0b0260 0x7fbacc13eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:44646/0 (socket says 172.21.15.187:44646) 2024-05-08T04:08:10.244 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.241+0000 7fbac3fff640 1 -- 172.21.15.187:0/3902082037 learned_addr learned my addr 172.21.15.187:0/3902082037 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-05-08T04:08:10.244 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.241+0000 7fbacb7fe640 1 --2- 172.21.15.187:0/3902082037 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbacc075ea0 0x7fbacc13e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:10.244 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.242+0000 7fbac3fff640 1 -- 172.21.15.187:0/3902082037 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbacc075ea0 msgr2=0x7fbacc13e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:10.244 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.242+0000 7fbac3fff640 1 --2- 172.21.15.187:0/3902082037 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbacc075ea0 0x7fbacc13e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.245 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.242+0000 7fbac3fff640 1 -- 172.21.15.187:0/3902082037 >> v1:172.21.15.170:6789/0 conn(0x7fbacc1415e0 legacy=0x7fbacc141a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:10.245 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.242+0000 7fbac3fff640 1 -- 172.21.15.187:0/3902082037 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbacc13f120 con 0x7fbacc0b0260 2024-05-08T04:08:10.246 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.242+0000 7fbac3fff640 1 --2- 172.21.15.187:0/3902082037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbacc0b0260 0x7fbacc13eb00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbab4009870 tx=0x7fbab402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2af562cbfcb6905 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:10.246 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.243+0000 7fbacaffd640 1 -- 172.21.15.187:0/3902082037 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab4034070 con 0x7fbacc0b0260 2024-05-08T04:08:10.246 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.243+0000 7fbacaffd640 1 -- 172.21.15.187:0/3902082037 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbab4030840 con 0x7fbacc0b0260 2024-05-08T04:08:10.246 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.243+0000 7fbacaffd640 1 -- 172.21.15.187:0/3902082037 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab4030c10 con 0x7fbacc0b0260 2024-05-08T04:08:10.246 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.243+0000 7fbad1eb5640 1 -- 172.21.15.187:0/3902082037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbacc0b0260 msgr2=0x7fbacc13eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:10.246 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.243+0000 7fbad1eb5640 1 --2- 172.21.15.187:0/3902082037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbacc0b0260 0x7fbacc13eb00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbab4009870 tx=0x7fbab402b980 comp rx=0 tx=0).stop 2024-05-08T04:08:10.246 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.243+0000 7fbad1eb5640 1 -- 172.21.15.187:0/3902082037 shutdown_connections 2024-05-08T04:08:10.246 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.243+0000 7fbad1eb5640 1 --2- 172.21.15.187:0/3902082037 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbacc0b0260 0x7fbacc13eb00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.246 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.243+0000 7fbad1eb5640 1 --2- 172.21.15.187:0/3902082037 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbacc075ea0 0x7fbacc13e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.246 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.243+0000 7fbad1eb5640 1 -- 172.21.15.187:0/3902082037 >> 172.21.15.187:0/3902082037 conn(0x7fbacc0b6020 msgr2=0x7fbacc0b6430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:10.246 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.243+0000 7fbad1eb5640 1 -- 172.21.15.187:0/3902082037 shutdown_connections 2024-05-08T04:08:10.247 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.243+0000 7fbad1eb5640 1 -- 172.21.15.187:0/3902082037 wait complete. 2024-05-08T04:08:10.249 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.244+0000 7fbad1eb5640 1 Processor -- start 2024-05-08T04:08:10.249 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.244+0000 7fbad1eb5640 1 -- start start 2024-05-08T04:08:10.249 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.245+0000 7fbad1eb5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbacc075ea0 0x7fbacc0724d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:10.249 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.245+0000 7fbad1eb5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbacc0b0260 0x7fbacc1d91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:10.249 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.245+0000 7fbad1eb5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbacc1415e0 0x7fbacc1452b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:10.249 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.245+0000 7fbad1eb5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbacc0b1790 con 0x7fbacc1415e0 2024-05-08T04:08:10.249 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.245+0000 7fbad1eb5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbacc0b1610 con 0x7fbacc0b0260 2024-05-08T04:08:10.249 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.245+0000 7fbad1eb5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fbacc14bef0 con 0x7fbacc075ea0 2024-05-08T04:08:10.249 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.245+0000 7fbac3fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbacc0b0260 0x7fbacc1d91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:10.249 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.245+0000 7fbac3fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbacc0b0260 0x7fbacc1d91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:44648/0 (socket says 172.21.15.187:44648) 2024-05-08T04:08:10.249 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.245+0000 7fbac3fff640 1 -- 172.21.15.187:0/1667339123 learned_addr learned my addr 172.21.15.187:0/1667339123 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-05-08T04:08:10.249 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.245+0000 7fbacbfff640 1 --2- 172.21.15.187:0/1667339123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbacc1415e0 0x7fbacc1452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:10.249 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.245+0000 7fbacb7fe640 1 --2- 172.21.15.187:0/1667339123 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbacc075ea0 0x7fbacc0724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:10.250 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.245+0000 7fbac3fff640 1 -- 172.21.15.187:0/1667339123 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbacc075ea0 msgr2=0x7fbacc0724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:10.250 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.245+0000 7fbac3fff640 1 --2- 172.21.15.187:0/1667339123 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbacc075ea0 0x7fbacc0724d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.250 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.245+0000 7fbac3fff640 1 -- 172.21.15.187:0/1667339123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbacc1415e0 msgr2=0x7fbacc1452b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:10.250 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.245+0000 7fbac3fff640 1 --2- 172.21.15.187:0/1667339123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbacc1415e0 0x7fbacc1452b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.250 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.245+0000 7fbac3fff640 1 -- 172.21.15.187:0/1667339123 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbacc142d50 con 0x7fbacc0b0260 2024-05-08T04:08:10.250 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.245+0000 7fbacb7fe640 1 --2- 172.21.15.187:0/1667339123 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbacc075ea0 0x7fbacc0724d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:08:10.250 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.245+0000 7fbac3fff640 1 --2- 172.21.15.187:0/1667339123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbacc0b0260 0x7fbacc1d91a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbab402b9c0 tx=0x7fbab4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:10.250 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.246+0000 7fbac8ff9640 1 -- 172.21.15.187:0/1667339123 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab4048070 con 0x7fbacc0b0260 2024-05-08T04:08:10.250 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.246+0000 7fbac8ff9640 1 -- 172.21.15.187:0/1667339123 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbab4030640 con 0x7fbacc0b0260 2024-05-08T04:08:10.250 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.246+0000 7fbad1eb5640 1 -- 172.21.15.187:0/1667339123 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbacc1457f0 con 0x7fbacc0b0260 2024-05-08T04:08:10.250 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.246+0000 7fbac8ff9640 1 -- 172.21.15.187:0/1667339123 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab4034040 con 0x7fbacc0b0260 2024-05-08T04:08:10.250 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.246+0000 7fbad1eb5640 1 -- 172.21.15.187:0/1667339123 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbacc0af130 con 0x7fbacc0b0260 2024-05-08T04:08:10.250 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.246+0000 7fbac8ff9640 1 -- 172.21.15.187:0/1667339123 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbab404b070 con 0x7fbacc0b0260 2024-05-08T04:08:10.250 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.246+0000 7fbac8ff9640 1 -- 172.21.15.187:0/1667339123 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fbab4043410 con 0x7fbacc0b0260 2024-05-08T04:08:10.250 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.247+0000 7fbad1eb5640 1 -- 172.21.15.187:0/1667339123 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbacc069bd0 con 0x7fbacc0b0260 2024-05-08T04:08:10.251 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.249+0000 7fbac8ff9640 1 -- 172.21.15.187:0/1667339123 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fbab4052070 con 0x7fbacc0b0260 2024-05-08T04:08:10.323 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.321+0000 7fbad1eb5640 1 -- 172.21.15.187:0/1667339123 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "66600f89-4532-43b1-85d3-6d8c736cfaf2", "id": 1} v 0) -- 0x7fbacc0b0ea0 con 0x7fbacc0b0260 2024-05-08T04:08:10.331 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.329+0000 7fbac8ff9640 1 -- 172.21.15.187:0/1667339123 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "66600f89-4532-43b1-85d3-6d8c736cfaf2", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fbab4030810 con 0x7fbacc0b0260 2024-05-08T04:08:10.331 INFO:teuthology.orchestra.run.smithi187.stdout:1 2024-05-08T04:08:10.332 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.330+0000 7fbad1eb5640 1 -- 172.21.15.187:0/1667339123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbacc0b0260 msgr2=0x7fbacc1d91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:10.332 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.330+0000 7fbad1eb5640 1 --2- 172.21.15.187:0/1667339123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbacc0b0260 0x7fbacc1d91a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbab402b9c0 tx=0x7fbab4004730 comp rx=0 tx=0).stop 2024-05-08T04:08:10.332 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.330+0000 7fbad1eb5640 1 -- 172.21.15.187:0/1667339123 shutdown_connections 2024-05-08T04:08:10.333 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.330+0000 7fbad1eb5640 1 --2- 172.21.15.187:0/1667339123 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbacc0b0260 0x7fbacc1d91a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.333 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.330+0000 7fbad1eb5640 1 --2- 172.21.15.187:0/1667339123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbacc1415e0 0x7fbacc1452b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.333 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.330+0000 7fbad1eb5640 1 --2- 172.21.15.187:0/1667339123 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbacc075ea0 0x7fbacc0724d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.333 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.330+0000 7fbad1eb5640 1 -- 172.21.15.187:0/1667339123 >> 172.21.15.187:0/1667339123 conn(0x7fbacc0b6020 msgr2=0x7fbacc073310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:10.333 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.330+0000 7fbad1eb5640 1 -- 172.21.15.187:0/1667339123 shutdown_connections 2024-05-08T04:08:10.333 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.330+0000 7fbad1eb5640 1 -- 172.21.15.187:0/1667339123 wait complete. 2024-05-08T04:08:10.343 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new a11608b2-0871-497c-aa38-264baea6bf39 2 2024-05-08T04:08:10.441 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.439+0000 7f3c95aff640 1 Processor -- start 2024-05-08T04:08:10.442 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.439+0000 7f3c95aff640 1 -- start start 2024-05-08T04:08:10.443 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.440+0000 7f3c95aff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3c9013ed40 0x7f3c901415c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:10.443 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.440+0000 7f3c95aff640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3c900b1090 0x7f3c90141b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:10.443 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.440+0000 7f3c95aff640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c900aa650 con 0x7f3c90075ea0 2024-05-08T04:08:10.443 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.440+0000 7f3c95aff640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7f3c90076280 con 0x7f3c900b1090 2024-05-08T04:08:10.443 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.440+0000 7f3c95aff640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f3c900b16a0 con 0x7f3c9013ed40 2024-05-08T04:08:10.443 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.440+0000 7f3c8f7fe640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f3c90075ea0 0x7f3c9013e630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.187:40872/0 (socket says 172.21.15.187:40872) 2024-05-08T04:08:10.443 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.440+0000 7f3c8f7fe640 1 -- 172.21.15.187:0/3671070871 learned_addr learned my addr 172.21.15.187:0/3671070871 (peer_addr_for_me v1:172.21.15.187:0/0) 2024-05-08T04:08:10.444 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.440+0000 7f3c8ffff640 1 --2- 172.21.15.187:0/3671070871 >> v2:172.21.15.170:3300/0 conn(0x7f3c900b1090 0x7f3c90141b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:10.444 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.440+0000 7f3c86ffd640 1 --2- 172.21.15.187:0/3671070871 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3c9013ed40 0x7f3c901415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:10.444 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.440+0000 7f3c8effd640 1 -- 172.21.15.187:0/3671070871 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2822349933 0 0) 0x7f3c900aa650 con 0x7f3c90075ea0 2024-05-08T04:08:10.444 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.440+0000 7f3c8effd640 1 -- 172.21.15.187:0/3671070871 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c7c003520 con 0x7f3c90075ea0 2024-05-08T04:08:10.444 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.441+0000 7f3c8ffff640 1 -- 172.21.15.187:0/3671070871 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3c9013ed40 msgr2=0x7f3c901415c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:10.444 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.441+0000 7f3c8ffff640 1 --2- 172.21.15.187:0/3671070871 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3c9013ed40 0x7f3c901415c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.444 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.441+0000 7f3c8ffff640 1 -- 172.21.15.187:0/3671070871 >> v1:172.21.15.187:6789/0 conn(0x7f3c90075ea0 legacy=0x7f3c9013e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:10.444 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.441+0000 7f3c8ffff640 1 -- 172.21.15.187:0/3671070871 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c90142040 con 0x7f3c900b1090 2024-05-08T04:08:10.444 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.441+0000 7f3c86ffd640 1 --2- 172.21.15.187:0/3671070871 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3c9013ed40 0x7f3c901415c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:08:10.444 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.441+0000 7f3c8ffff640 1 --2- 172.21.15.187:0/3671070871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c900b1090 0x7f3c90141b00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3c88009d90 tx=0x7f3c8802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a3fb771314c8513 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:10.447 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.442+0000 7f3c8effd640 1 -- 172.21.15.187:0/3671070871 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c88039070 con 0x7f3c900b1090 2024-05-08T04:08:10.447 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.442+0000 7f3c8effd640 1 -- 172.21.15.187:0/3671070871 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c8802cc50 con 0x7f3c900b1090 2024-05-08T04:08:10.448 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.442+0000 7f3c8effd640 1 -- 172.21.15.187:0/3671070871 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c88034440 con 0x7f3c900b1090 2024-05-08T04:08:10.448 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.442+0000 7f3c95aff640 1 -- 172.21.15.187:0/3671070871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c900b1090 msgr2=0x7f3c90141b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:10.448 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.442+0000 7f3c95aff640 1 --2- 172.21.15.187:0/3671070871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c900b1090 0x7f3c90141b00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3c88009d90 tx=0x7f3c8802bea0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.448 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.442+0000 7f3c95aff640 1 -- 172.21.15.187:0/3671070871 shutdown_connections 2024-05-08T04:08:10.448 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.442+0000 7f3c95aff640 1 --2- 172.21.15.187:0/3671070871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c900b1090 0x7f3c90141b00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.448 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.442+0000 7f3c95aff640 1 --2- 172.21.15.187:0/3671070871 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3c9013ed40 0x7f3c901415c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.448 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.442+0000 7f3c95aff640 1 -- 172.21.15.187:0/3671070871 >> 172.21.15.187:0/3671070871 conn(0x7f3c900b6020 msgr2=0x7f3c900b6430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:10.448 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.442+0000 7f3c95aff640 1 -- 172.21.15.187:0/3671070871 shutdown_connections 2024-05-08T04:08:10.448 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.442+0000 7f3c95aff640 1 -- 172.21.15.187:0/3671070871 wait complete. 2024-05-08T04:08:10.448 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.443+0000 7f3c95aff640 1 Processor -- start 2024-05-08T04:08:10.448 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.443+0000 7f3c95aff640 1 -- start start 2024-05-08T04:08:10.448 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.444+0000 7f3c95aff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3c90075ea0 0x7f3c90138360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:10.448 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.444+0000 7f3c95aff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c900b1090 0x7f3c901368e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:10.448 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.444+0000 7f3c95aff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c9013ed40 0x7f3c901da050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:10.448 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.444+0000 7f3c95aff640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9014bed0 con 0x7f3c900b1090 2024-05-08T04:08:10.449 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.444+0000 7f3c95aff640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3c900b04b0 con 0x7f3c9013ed40 2024-05-08T04:08:10.449 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.444+0000 7f3c95aff640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f3c9014c050 con 0x7f3c90075ea0 2024-05-08T04:08:10.449 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.444+0000 7f3c8ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c9013ed40 0x7f3c901da050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:10.449 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.444+0000 7f3c8ffff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c9013ed40 0x7f3c901da050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:44656/0 (socket says 172.21.15.187:44656) 2024-05-08T04:08:10.449 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.444+0000 7f3c8ffff640 1 -- 172.21.15.187:0/2610250053 learned_addr learned my addr 172.21.15.187:0/2610250053 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-05-08T04:08:10.449 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.444+0000 7f3c8f7fe640 1 --2- 172.21.15.187:0/2610250053 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3c90075ea0 0x7f3c90138360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:10.449 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.444+0000 7f3c86ffd640 1 --2- 172.21.15.187:0/2610250053 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c900b1090 0x7f3c901368e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:10.449 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.444+0000 7f3c8ffff640 1 -- 172.21.15.187:0/2610250053 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3c90075ea0 msgr2=0x7f3c90138360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:10.449 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.444+0000 7f3c8ffff640 1 --2- 172.21.15.187:0/2610250053 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3c90075ea0 0x7f3c90138360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.449 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.444+0000 7f3c8ffff640 1 -- 172.21.15.187:0/2610250053 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c900b1090 msgr2=0x7f3c901368e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:10.449 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.444+0000 7f3c8ffff640 1 --2- 172.21.15.187:0/2610250053 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c900b1090 0x7f3c901368e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.449 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.444+0000 7f3c8ffff640 1 -- 172.21.15.187:0/2610250053 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c90137c60 con 0x7f3c9013ed40 2024-05-08T04:08:10.449 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.444+0000 7f3c8ffff640 1 --2- 172.21.15.187:0/2610250053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c9013ed40 0x7f3c901da050 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3c88009d90 tx=0x7f3c88004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:10.449 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.444+0000 7f3c8cff9640 1 -- 172.21.15.187:0/2610250053 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c88044020 con 0x7f3c9013ed40 2024-05-08T04:08:10.450 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.445+0000 7f3c8cff9640 1 -- 172.21.15.187:0/2610250053 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c8803a040 con 0x7f3c9013ed40 2024-05-08T04:08:10.450 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.445+0000 7f3c95aff640 1 -- 172.21.15.187:0/2610250053 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c90137f40 con 0x7f3c9013ed40 2024-05-08T04:08:10.450 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.445+0000 7f3c8cff9640 1 -- 172.21.15.187:0/2610250053 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c88039050 con 0x7f3c9013ed40 2024-05-08T04:08:10.450 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.445+0000 7f3c95aff640 1 -- 172.21.15.187:0/2610250053 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c900afac0 con 0x7f3c9013ed40 2024-05-08T04:08:10.450 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.445+0000 7f3c8cff9640 1 -- 172.21.15.187:0/2610250053 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3c88003990 con 0x7f3c9013ed40 2024-05-08T04:08:10.450 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.445+0000 7f3c8cff9640 1 -- 172.21.15.187:0/2610250053 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f3c88004670 con 0x7f3c9013ed40 2024-05-08T04:08:10.450 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.445+0000 7f3c95aff640 1 -- 172.21.15.187:0/2610250053 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c90131e40 con 0x7f3c9013ed40 2024-05-08T04:08:10.450 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.448+0000 7f3c8cff9640 1 -- 172.21.15.187:0/2610250053 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f3c90131e40 con 0x7f3c9013ed40 2024-05-08T04:08:10.524 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.522+0000 7f3c95aff640 1 -- 172.21.15.187:0/2610250053 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a11608b2-0871-497c-aa38-264baea6bf39", "id": 2} v 0) -- 0x7f3c90069bd0 con 0x7f3c9013ed40 2024-05-08T04:08:10.531 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.530+0000 7f3c8cff9640 1 -- 172.21.15.187:0/2610250053 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a11608b2-0871-497c-aa38-264baea6bf39", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f3c8804e060 con 0x7f3c9013ed40 2024-05-08T04:08:10.532 INFO:teuthology.orchestra.run.smithi187.stdout:2 2024-05-08T04:08:10.533 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.530+0000 7f3c95aff640 1 -- 172.21.15.187:0/2610250053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c9013ed40 msgr2=0x7f3c901da050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:10.533 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.530+0000 7f3c95aff640 1 --2- 172.21.15.187:0/2610250053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c9013ed40 0x7f3c901da050 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3c88009d90 tx=0x7f3c88004a40 comp rx=0 tx=0).stop 2024-05-08T04:08:10.533 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.531+0000 7f3c95aff640 1 -- 172.21.15.187:0/2610250053 shutdown_connections 2024-05-08T04:08:10.533 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.531+0000 7f3c95aff640 1 --2- 172.21.15.187:0/2610250053 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c9013ed40 0x7f3c901da050 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.533 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.531+0000 7f3c95aff640 1 --2- 172.21.15.187:0/2610250053 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c900b1090 0x7f3c901368e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.533 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.531+0000 7f3c95aff640 1 --2- 172.21.15.187:0/2610250053 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3c90075ea0 0x7f3c90138360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.533 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.531+0000 7f3c95aff640 1 -- 172.21.15.187:0/2610250053 >> 172.21.15.187:0/2610250053 conn(0x7f3c900b6020 msgr2=0x7f3c900b6400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:10.533 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.531+0000 7f3c95aff640 1 -- 172.21.15.187:0/2610250053 shutdown_connections 2024-05-08T04:08:10.533 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.531+0000 7f3c95aff640 1 -- 172.21.15.187:0/2610250053 wait complete. 2024-05-08T04:08:10.543 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 8caae6f8-790c-426c-af8a-3a9ef54eae48 3 2024-05-08T04:08:10.642 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.640+0000 7f244d4e3640 1 Processor -- start 2024-05-08T04:08:10.642 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.640+0000 7f244d4e3640 1 -- start start 2024-05-08T04:08:10.648 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.641+0000 7f244d4e3640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f24480af1c0 0x7f2448147810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:10.649 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.641+0000 7f244d4e3640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2448147d50 con 0x7f2448068a10 2024-05-08T04:08:10.649 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.641+0000 7f244d4e3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24480af680 con 0x7f2448068df0 2024-05-08T04:08:10.649 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.641+0000 7f244d4e3640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7f2448071230 con 0x7f24480af1c0 2024-05-08T04:08:10.649 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.641+0000 7f24467fc640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f2448068df0 0x7f2448147100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.187:40876/0 (socket says 172.21.15.187:40876) 2024-05-08T04:08:10.649 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.641+0000 7f24467fc640 1 -- 172.21.15.187:0/805849618 learned_addr learned my addr 172.21.15.187:0/805849618 (peer_addr_for_me v1:172.21.15.187:0/0) 2024-05-08T04:08:10.649 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.641+0000 7f24477fe640 1 --2- 172.21.15.187:0/805849618 >> v2:172.21.15.170:3300/0 conn(0x7f24480af1c0 0x7f2448147810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:10.649 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.641+0000 7f2445ffb640 1 -- 172.21.15.187:0/805849618 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2600145715 0 0) 0x7f24480af680 con 0x7f2448068df0 2024-05-08T04:08:10.649 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.641+0000 7f2445ffb640 1 -- 172.21.15.187:0/805849618 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2430003520 con 0x7f2448068df0 2024-05-08T04:08:10.649 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.641+0000 7f2445ffb640 1 -- 172.21.15.187:0/805849618 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1694597244 0 0) 0x7f2448147d50 con 0x7f2448068a10 2024-05-08T04:08:10.649 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.641+0000 7f2445ffb640 1 -- 172.21.15.187:0/805849618 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24480af680 con 0x7f2448068a10 2024-05-08T04:08:10.649 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.641+0000 7f2445ffb640 1 -- 172.21.15.187:0/805849618 <== mon.1 v1:172.21.15.187:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3650566040 0 0) 0x7f2430003520 con 0x7f2448068df0 2024-05-08T04:08:10.649 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.641+0000 7f2445ffb640 1 -- 172.21.15.187:0/805849618 >> v2:172.21.15.170:3300/0 conn(0x7f24480af1c0 msgr2=0x7f2448147810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:10.649 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.641+0000 7f2445ffb640 1 --2- 172.21.15.187:0/805849618 >> v2:172.21.15.170:3300/0 conn(0x7f24480af1c0 0x7f2448147810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.649 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.641+0000 7f2445ffb640 1 -- 172.21.15.187:0/805849618 >> v1:172.21.15.170:6789/0 conn(0x7f2448068a10 legacy=0x7f2448074130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:10.650 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.641+0000 7f2445ffb640 1 -- 172.21.15.187:0/805849618 --> v1:172.21.15.187:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2448071510 con 0x7f2448068df0 2024-05-08T04:08:10.650 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.641+0000 7f2445ffb640 1 -- 172.21.15.187:0/805849618 <== mon.1 v1:172.21.15.187:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1210551688 0 0) 0x7f243c0016f0 con 0x7f2448068df0 2024-05-08T04:08:10.650 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.641+0000 7f24477fe640 1 --2- 172.21.15.187:0/805849618 >> v2:172.21.15.170:3300/0 conn(0x7f24480af1c0 0x7f2448147810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:08:10.650 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.641+0000 7f2445ffb640 1 -- 172.21.15.187:0/805849618 >> v1:172.21.15.187:6789/0 conn(0x7f2448068df0 legacy=0x7f2448147100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:10.650 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.641+0000 7f2445ffb640 1 --2- 172.21.15.187:0/805849618 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2430003c30 0x7f2430004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:10.650 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.641+0000 7f2445ffb640 1 --2- 172.21.15.187:0/805849618 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2430004800 0x7f2430004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:10.650 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.641+0000 7f2445ffb640 1 --2- 172.21.15.187:0/805849618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2430005280 0x7f243000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:10.650 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.641+0000 7f2445ffb640 1 -- 172.21.15.187:0/805849618 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2448071230 con 0x7f2430005280 2024-05-08T04:08:10.650 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.641+0000 7f2445ffb640 1 -- 172.21.15.187:0/805849618 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f243000c1c0 con 0x7f2430004800 2024-05-08T04:08:10.650 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.641+0000 7f2445ffb640 1 -- 172.21.15.187:0/805849618 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f243000c3c0 con 0x7f2430003c30 2024-05-08T04:08:10.650 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.641+0000 7f24467fc640 1 --2- 172.21.15.187:0/805849618 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2430004800 0x7f2430004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:10.650 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.642+0000 7f24477fe640 1 --2- 172.21.15.187:0/805849618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2430005280 0x7f243000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:10.650 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.642+0000 7f2446ffd640 1 --2- 172.21.15.187:0/805849618 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2430003c30 0x7f2430004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:10.650 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.642+0000 7f24467fc640 1 -- 172.21.15.187:0/805849618 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2430003c30 msgr2=0x7f2430004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:10.651 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.642+0000 7f24467fc640 1 --2- 172.21.15.187:0/805849618 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2430003c30 0x7f2430004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.651 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.642+0000 7f24467fc640 1 -- 172.21.15.187:0/805849618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2430005280 msgr2=0x7f243000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:10.651 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.642+0000 7f24467fc640 1 --2- 172.21.15.187:0/805849618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2430005280 0x7f243000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.651 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.642+0000 7f24467fc640 1 -- 172.21.15.187:0/805849618 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2430003520 con 0x7f2430004800 2024-05-08T04:08:10.651 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.642+0000 7f24477fe640 1 -- 172.21.15.187:0/805849618 reap_dead start 2024-05-08T04:08:10.651 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.642+0000 7f24467fc640 1 --2- 172.21.15.187:0/805849618 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2430004800 0x7f2430004c80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f244806ff90 tx=0x7f2430005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98dd1e5c56768495 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:10.651 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.643+0000 7f2445ffb640 1 -- 172.21.15.187:0/805849618 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c007ca0 con 0x7f2430004800 2024-05-08T04:08:10.651 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.643+0000 7f2445ffb640 1 -- 172.21.15.187:0/805849618 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f243c030070 con 0x7f2430004800 2024-05-08T04:08:10.651 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.643+0000 7f2445ffb640 1 -- 172.21.15.187:0/805849618 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c005d70 con 0x7f2430004800 2024-05-08T04:08:10.651 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.643+0000 7f244d4e3640 1 -- 172.21.15.187:0/805849618 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2430004800 msgr2=0x7f2430004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:10.651 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.643+0000 7f244d4e3640 1 --2- 172.21.15.187:0/805849618 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2430004800 0x7f2430004c80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f244806ff90 tx=0x7f2430005830 comp rx=0 tx=0).stop 2024-05-08T04:08:10.651 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.643+0000 7f244d4e3640 1 -- 172.21.15.187:0/805849618 shutdown_connections 2024-05-08T04:08:10.651 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.643+0000 7f244d4e3640 1 --2- 172.21.15.187:0/805849618 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2430004800 0x7f2430004c80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.651 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.643+0000 7f244d4e3640 1 -- 172.21.15.187:0/805849618 >> 172.21.15.187:0/805849618 conn(0x7f24480b5c20 msgr2=0x7f24480b20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:10.652 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.643+0000 7f244d4e3640 1 -- 172.21.15.187:0/805849618 shutdown_connections 2024-05-08T04:08:10.652 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.643+0000 7f244d4e3640 1 -- 172.21.15.187:0/805849618 wait complete. 2024-05-08T04:08:10.652 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.644+0000 7f244d4e3640 1 Processor -- start 2024-05-08T04:08:10.652 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.644+0000 7f244d4e3640 1 -- start start 2024-05-08T04:08:10.652 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.645+0000 7f244d4e3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2430004800 0x7f2448140980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:10.652 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.645+0000 7f244d4e3640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2448146830 0x7f244813def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:10.652 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.645+0000 7f244d4e3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f244813e430 0x7f24480b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:10.652 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.645+0000 7f244d4e3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2448074130 con 0x7f244813e430 2024-05-08T04:08:10.652 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.645+0000 7f244d4e3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f24480afcd0 con 0x7f2430004800 2024-05-08T04:08:10.652 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.645+0000 7f244d4e3640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f24480742b0 con 0x7f2448146830 2024-05-08T04:08:10.652 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.645+0000 7f2446ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2430004800 0x7f2448140980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:10.652 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.645+0000 7f2446ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2430004800 0x7f2448140980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:44672/0 (socket says 172.21.15.187:44672) 2024-05-08T04:08:10.652 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.645+0000 7f2446ffd640 1 -- 172.21.15.187:0/1537603704 learned_addr learned my addr 172.21.15.187:0/1537603704 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-05-08T04:08:10.652 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.645+0000 7f24467fc640 1 --2- 172.21.15.187:0/1537603704 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2448146830 0x7f244813def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:10.653 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.645+0000 7f24477fe640 1 --2- 172.21.15.187:0/1537603704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f244813e430 0x7f24480b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:10.653 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.645+0000 7f2446ffd640 1 -- 172.21.15.187:0/1537603704 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2448146830 msgr2=0x7f244813def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:10.653 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.645+0000 7f2446ffd640 1 --2- 172.21.15.187:0/1537603704 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2448146830 0x7f244813def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.653 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.645+0000 7f2446ffd640 1 -- 172.21.15.187:0/1537603704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f244813e430 msgr2=0x7f24480b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:10.653 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.645+0000 7f2446ffd640 1 --2- 172.21.15.187:0/1537603704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f244813e430 0x7f24480b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.653 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.645+0000 7f2446ffd640 1 -- 172.21.15.187:0/1537603704 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f244813d6b0 con 0x7f2430004800 2024-05-08T04:08:10.653 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.645+0000 7f2446ffd640 1 --2- 172.21.15.187:0/1537603704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2430004800 0x7f2448140980 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f243800f850 tx=0x7f243800fd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:10.653 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.645+0000 7f242f7fe640 1 -- 172.21.15.187:0/1537603704 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24380052e0 con 0x7f2430004800 2024-05-08T04:08:10.653 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.646+0000 7f242f7fe640 1 -- 172.21.15.187:0/1537603704 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2438018070 con 0x7f2430004800 2024-05-08T04:08:10.653 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.646+0000 7f242f7fe640 1 -- 172.21.15.187:0/1537603704 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243800d970 con 0x7f2430004800 2024-05-08T04:08:10.653 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.646+0000 7f244d4e3640 1 -- 172.21.15.187:0/1537603704 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24480b5210 con 0x7f2430004800 2024-05-08T04:08:10.653 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.646+0000 7f244d4e3640 1 -- 172.21.15.187:0/1537603704 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f244813f040 con 0x7f2430004800 2024-05-08T04:08:10.653 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.646+0000 7f242f7fe640 1 -- 172.21.15.187:0/1537603704 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f243800db10 con 0x7f2430004800 2024-05-08T04:08:10.653 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.646+0000 7f242f7fe640 1 -- 172.21.15.187:0/1537603704 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f2438003d20 con 0x7f2430004800 2024-05-08T04:08:10.653 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.646+0000 7f244d4e3640 1 -- 172.21.15.187:0/1537603704 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2448146500 con 0x7f2430004800 2024-05-08T04:08:10.654 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.648+0000 7f242f7fe640 1 -- 172.21.15.187:0/1537603704 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f24380035f0 con 0x7f2430004800 2024-05-08T04:08:10.722 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.720+0000 7f244d4e3640 1 -- 172.21.15.187:0/1537603704 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8caae6f8-790c-426c-af8a-3a9ef54eae48", "id": 3} v 0) -- 0x7f2448069bd0 con 0x7f2430004800 2024-05-08T04:08:10.730 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-08T04:08:10.727+0000 7f80f120d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T04:08:10.730 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.728+0000 7f242f7fe640 1 -- 172.21.15.187:0/1537603704 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8caae6f8-790c-426c-af8a-3a9ef54eae48", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f24380037d0 con 0x7f2430004800 2024-05-08T04:08:10.730 INFO:teuthology.orchestra.run.smithi187.stdout:3 2024-05-08T04:08:10.731 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.729+0000 7f244d4e3640 1 -- 172.21.15.187:0/1537603704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2430004800 msgr2=0x7f2448140980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:10.731 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.729+0000 7f244d4e3640 1 --2- 172.21.15.187:0/1537603704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2430004800 0x7f2448140980 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f243800f850 tx=0x7f243800fd10 comp rx=0 tx=0).stop 2024-05-08T04:08:10.731 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.729+0000 7f244d4e3640 1 -- 172.21.15.187:0/1537603704 shutdown_connections 2024-05-08T04:08:10.731 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.729+0000 7f244d4e3640 1 --2- 172.21.15.187:0/1537603704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f244813e430 0x7f24480b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.731 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.729+0000 7f244d4e3640 1 --2- 172.21.15.187:0/1537603704 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2448146830 0x7f244813def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.731 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.729+0000 7f244d4e3640 1 --2- 172.21.15.187:0/1537603704 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2430004800 0x7f2448140980 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.731 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.729+0000 7f244d4e3640 1 -- 172.21.15.187:0/1537603704 >> 172.21.15.187:0/1537603704 conn(0x7f24480b5c20 msgr2=0x7f2448138d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:10.731 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.729+0000 7f244d4e3640 1 -- 172.21.15.187:0/1537603704 shutdown_connections 2024-05-08T04:08:10.732 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.729+0000 7f244d4e3640 1 -- 172.21.15.187:0/1537603704 wait complete. 2024-05-08T04:08:10.741 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 0e804208-3607-4d71-b771-f79fb27e7968 4 2024-05-08T04:08:10.822 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-08T04:08:10.820+0000 7f80f120d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T04:08:10.832 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-05-08T04:08:10.831+0000 7f7ca884b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T04:08:10.839 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.837+0000 7fb17013f640 1 Processor -- start 2024-05-08T04:08:10.839 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.838+0000 7fb17013f640 1 -- start start 2024-05-08T04:08:10.841 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.838+0000 7fb17013f640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fb168073fd0 0x7fb168071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:10.841 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.838+0000 7fb17013f640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb168140440 0x7fb1680af600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:10.841 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.838+0000 7fb17013f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb168073280 con 0x7fb1680afb40 2024-05-08T04:08:10.841 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.838+0000 7fb17013f640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7fb168071a70 con 0x7fb168073fd0 2024-05-08T04:08:10.841 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.838+0000 7fb17013f640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fb168073050 con 0x7fb168140440 2024-05-08T04:08:10.841 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.838+0000 7fb16e6dc640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fb1680afb40 0x7fb1681439c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.187:40878/0 (socket says 172.21.15.187:40878) 2024-05-08T04:08:10.842 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.838+0000 7fb16e6dc640 1 -- 172.21.15.187:0/3046185412 learned_addr learned my addr 172.21.15.187:0/3046185412 (peer_addr_for_me v1:172.21.15.187:0/0) 2024-05-08T04:08:10.842 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.838+0000 7fb16dedb640 1 --2- 172.21.15.187:0/3046185412 >> v2:172.21.15.170:3300/0 conn(0x7fb168073fd0 0x7fb168071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:10.842 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.838+0000 7fb16d6da640 1 --2- 172.21.15.187:0/3046185412 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb168140440 0x7fb1680af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:10.842 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.839+0000 7fb16ced9640 1 -- 172.21.15.187:0/3046185412 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1287840204 0 0) 0x7fb168073280 con 0x7fb1680afb40 2024-05-08T04:08:10.842 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.839+0000 7fb16ced9640 1 -- 172.21.15.187:0/3046185412 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb154003520 con 0x7fb1680afb40 2024-05-08T04:08:10.842 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.839+0000 7fb16dedb640 1 -- 172.21.15.187:0/3046185412 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb168140440 msgr2=0x7fb1680af600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:10.842 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.839+0000 7fb16dedb640 1 --2- 172.21.15.187:0/3046185412 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb168140440 0x7fb1680af600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.842 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.839+0000 7fb16dedb640 1 -- 172.21.15.187:0/3046185412 >> v1:172.21.15.187:6789/0 conn(0x7fb1680afb40 legacy=0x7fb1681439c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:10.842 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.839+0000 7fb16dedb640 1 -- 172.21.15.187:0/3046185412 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb168073560 con 0x7fb168073fd0 2024-05-08T04:08:10.842 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.839+0000 7fb16d6da640 1 --2- 172.21.15.187:0/3046185412 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb168140440 0x7fb1680af600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:08:10.846 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.839+0000 7fb16dedb640 1 --2- 172.21.15.187:0/3046185412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb168073fd0 0x7fb168071530 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb1580097a0 tx=0x7fb15802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20311cea6ffebdf5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:10.846 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.840+0000 7fb16ced9640 1 -- 172.21.15.187:0/3046185412 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb158035020 con 0x7fb168073fd0 2024-05-08T04:08:10.846 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.840+0000 7fb16ced9640 1 -- 172.21.15.187:0/3046185412 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb158031810 con 0x7fb168073fd0 2024-05-08T04:08:10.846 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.840+0000 7fb16ced9640 1 -- 172.21.15.187:0/3046185412 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb158031be0 con 0x7fb168073fd0 2024-05-08T04:08:10.846 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.840+0000 7fb17013f640 1 -- 172.21.15.187:0/3046185412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb168073fd0 msgr2=0x7fb168071530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:10.846 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.840+0000 7fb17013f640 1 --2- 172.21.15.187:0/3046185412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb168073fd0 0x7fb168071530 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb1580097a0 tx=0x7fb15802b910 comp rx=0 tx=0).stop 2024-05-08T04:08:10.846 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.840+0000 7fb17013f640 1 -- 172.21.15.187:0/3046185412 shutdown_connections 2024-05-08T04:08:10.846 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.840+0000 7fb17013f640 1 --2- 172.21.15.187:0/3046185412 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb168140440 0x7fb1680af600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.846 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.840+0000 7fb17013f640 1 --2- 172.21.15.187:0/3046185412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb168073fd0 0x7fb168071530 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.846 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.840+0000 7fb17013f640 1 -- 172.21.15.187:0/3046185412 >> 172.21.15.187:0/3046185412 conn(0x7fb1680b6020 msgr2=0x7fb1680b6430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:10.846 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.840+0000 7fb17013f640 1 -- 172.21.15.187:0/3046185412 shutdown_connections 2024-05-08T04:08:10.846 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.840+0000 7fb17013f640 1 -- 172.21.15.187:0/3046185412 wait complete. 2024-05-08T04:08:10.847 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.841+0000 7fb17013f640 1 Processor -- start 2024-05-08T04:08:10.847 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.841+0000 7fb17013f640 1 -- start start 2024-05-08T04:08:10.847 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.842+0000 7fb17013f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb168073fd0 0x7fb168075ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:10.847 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.842+0000 7fb17013f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1680afb40 0x7fb1680aa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:10.847 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.842+0000 7fb17013f640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb168140440 0x7fb1681363b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:10.847 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.842+0000 7fb17013f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb1680720b0 con 0x7fb168073fd0 2024-05-08T04:08:10.847 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.842+0000 7fb17013f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb1680710a0 con 0x7fb1680afb40 2024-05-08T04:08:10.847 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.842+0000 7fb17013f640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fb1680743b0 con 0x7fb168140440 2024-05-08T04:08:10.847 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.842+0000 7fb16d6da640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1680afb40 0x7fb1680aa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:10.847 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.842+0000 7fb16d6da640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1680afb40 0x7fb1680aa4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:44682/0 (socket says 172.21.15.187:44682) 2024-05-08T04:08:10.847 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.842+0000 7fb16d6da640 1 -- 172.21.15.187:0/785957977 learned_addr learned my addr 172.21.15.187:0/785957977 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-05-08T04:08:10.847 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.842+0000 7fb16dedb640 1 --2- 172.21.15.187:0/785957977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb168073fd0 0x7fb168075ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:10.847 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.842+0000 7fb16e6dc640 1 --2- 172.21.15.187:0/785957977 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb168140440 0x7fb1681363b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:10.847 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.842+0000 7fb16d6da640 1 -- 172.21.15.187:0/785957977 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb168140440 msgr2=0x7fb1681363b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:10.847 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.842+0000 7fb16d6da640 1 --2- 172.21.15.187:0/785957977 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb168140440 0x7fb1681363b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.848 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.842+0000 7fb16d6da640 1 -- 172.21.15.187:0/785957977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb168073fd0 msgr2=0x7fb168075ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:10.848 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.842+0000 7fb16d6da640 1 --2- 172.21.15.187:0/785957977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb168073fd0 0x7fb168075ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.848 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.842+0000 7fb16d6da640 1 -- 172.21.15.187:0/785957977 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1680ac940 con 0x7fb1680afb40 2024-05-08T04:08:10.848 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.843+0000 7fb16d6da640 1 --2- 172.21.15.187:0/785957977 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1680afb40 0x7fb1680aa4b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb16400e990 tx=0x7fb16400ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:10.848 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.843+0000 7fb15e7fc640 1 -- 172.21.15.187:0/785957977 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb164017070 con 0x7fb1680afb40 2024-05-08T04:08:10.848 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.843+0000 7fb15e7fc640 1 -- 172.21.15.187:0/785957977 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb16400c9e0 con 0x7fb1680afb40 2024-05-08T04:08:10.848 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.843+0000 7fb15e7fc640 1 -- 172.21.15.187:0/785957977 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16400cdb0 con 0x7fb1680afb40 2024-05-08T04:08:10.848 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.843+0000 7fb17013f640 1 -- 172.21.15.187:0/785957977 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1680aab80 con 0x7fb1680afb40 2024-05-08T04:08:10.848 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.843+0000 7fb17013f640 1 -- 172.21.15.187:0/785957977 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1680ab900 con 0x7fb1680afb40 2024-05-08T04:08:10.848 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.843+0000 7fb15e7fc640 1 -- 172.21.15.187:0/785957977 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb16401a070 con 0x7fb1680afb40 2024-05-08T04:08:10.848 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.843+0000 7fb15e7fc640 1 -- 172.21.15.187:0/785957977 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fb16401c960 con 0x7fb1680afb40 2024-05-08T04:08:10.848 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.844+0000 7fb17013f640 1 -- 172.21.15.187:0/785957977 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb168069bd0 con 0x7fb1680afb40 2024-05-08T04:08:10.849 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.846+0000 7fb15e7fc640 1 -- 172.21.15.187:0/785957977 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fb1680ab900 con 0x7fb1680afb40 2024-05-08T04:08:10.867 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-08T04:08:10.865+0000 7f80f120d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T04:08:10.919 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.917+0000 7fb17013f640 1 -- 172.21.15.187:0/785957977 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0e804208-3607-4d71-b771-f79fb27e7968", "id": 4} v 0) -- 0x7fb1680b5b60 con 0x7fb1680afb40 2024-05-08T04:08:10.922 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-05-08T04:08:10.920+0000 7f7ca884b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T04:08:10.926 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.924+0000 7fb15e7fc640 1 -- 172.21.15.187:0/785957977 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0e804208-3607-4d71-b771-f79fb27e7968", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fb16401cc00 con 0x7fb1680afb40 2024-05-08T04:08:10.926 INFO:teuthology.orchestra.run.smithi187.stdout:4 2024-05-08T04:08:10.927 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.925+0000 7fb17013f640 1 -- 172.21.15.187:0/785957977 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1680afb40 msgr2=0x7fb1680aa4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:10.928 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.925+0000 7fb17013f640 1 --2- 172.21.15.187:0/785957977 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1680afb40 0x7fb1680aa4b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb16400e990 tx=0x7fb16400ee50 comp rx=0 tx=0).stop 2024-05-08T04:08:10.928 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.925+0000 7fb17013f640 1 -- 172.21.15.187:0/785957977 shutdown_connections 2024-05-08T04:08:10.928 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.925+0000 7fb17013f640 1 --2- 172.21.15.187:0/785957977 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1680afb40 0x7fb1680aa4b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.928 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.925+0000 7fb17013f640 1 --2- 172.21.15.187:0/785957977 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb168140440 0x7fb1681363b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.928 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.925+0000 7fb17013f640 1 --2- 172.21.15.187:0/785957977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb168073fd0 0x7fb168075ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:10.928 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.925+0000 7fb17013f640 1 -- 172.21.15.187:0/785957977 >> 172.21.15.187:0/785957977 conn(0x7fb1680b6020 msgr2=0x7fb1680b6400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:10.928 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.925+0000 7fb17013f640 1 -- 172.21.15.187:0/785957977 shutdown_connections 2024-05-08T04:08:10.928 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:10.925+0000 7fb17013f640 1 -- 172.21.15.187:0/785957977 wait complete. 2024-05-08T04:08:10.938 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new f88f88db-216e-4acc-ba5a-89d91b98aeb4 5 2024-05-08T04:08:10.964 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-05-08T04:08:10.963+0000 7f7ca884b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T04:08:11.035 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.033+0000 7f00753ab640 1 Processor -- start 2024-05-08T04:08:11.035 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.033+0000 7f00753ab640 1 -- start start 2024-05-08T04:08:11.037 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.034+0000 7f00753ab640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f007013ed40 0x7f00701415c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:11.038 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.034+0000 7f00753ab640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00700aa650 con 0x7f00700b1090 2024-05-08T04:08:11.038 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.034+0000 7f00753ab640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0070142210 con 0x7f0070075ea0 2024-05-08T04:08:11.038 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.034+0000 7f00753ab640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7f0070076280 con 0x7f007013ed40 2024-05-08T04:08:11.038 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.034+0000 7f006effd640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f0070075ea0 0x7f007013e630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.187:40886/0 (socket says 172.21.15.187:40886) 2024-05-08T04:08:11.038 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.034+0000 7f006effd640 1 -- 172.21.15.187:0/1257013839 learned_addr learned my addr 172.21.15.187:0/1257013839 (peer_addr_for_me v1:172.21.15.187:0/0) 2024-05-08T04:08:11.038 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.034+0000 7f006e7fc640 1 --2- 172.21.15.187:0/1257013839 >> v2:172.21.15.170:3300/0 conn(0x7f007013ed40 0x7f00701415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:11.038 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.034+0000 7f006dffb640 1 -- 172.21.15.187:0/1257013839 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2547348607 0 0) 0x7f0070142210 con 0x7f0070075ea0 2024-05-08T04:08:11.038 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.034+0000 7f006dffb640 1 -- 172.21.15.187:0/1257013839 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0058003520 con 0x7f0070075ea0 2024-05-08T04:08:11.038 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.034+0000 7f006dffb640 1 -- 172.21.15.187:0/1257013839 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2727720849 0 0) 0x7f00700aa650 con 0x7f00700b1090 2024-05-08T04:08:11.038 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.034+0000 7f006dffb640 1 -- 172.21.15.187:0/1257013839 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0070142210 con 0x7f00700b1090 2024-05-08T04:08:11.038 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.034+0000 7f006e7fc640 1 -- 172.21.15.187:0/1257013839 >> v1:172.21.15.187:6789/0 conn(0x7f0070075ea0 legacy=0x7f007013e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:11.038 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.035+0000 7f006e7fc640 1 -- 172.21.15.187:0/1257013839 >> v1:172.21.15.170:6789/0 conn(0x7f00700b1090 legacy=0x7f0070141b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:11.039 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.035+0000 7f006e7fc640 1 -- 172.21.15.187:0/1257013839 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00700b0260 con 0x7f007013ed40 2024-05-08T04:08:11.039 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.035+0000 7f006e7fc640 1 --2- 172.21.15.187:0/1257013839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f007013ed40 0x7f00701415c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f00600097a0 tx=0x7f006002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca19d29ba25b43a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:11.039 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.035+0000 7f006dffb640 1 -- 172.21.15.187:0/1257013839 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0060035020 con 0x7f007013ed40 2024-05-08T04:08:11.039 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.035+0000 7f006dffb640 1 -- 172.21.15.187:0/1257013839 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0060031830 con 0x7f007013ed40 2024-05-08T04:08:11.039 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.035+0000 7f006dffb640 1 -- 172.21.15.187:0/1257013839 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0060031c00 con 0x7f007013ed40 2024-05-08T04:08:11.040 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.036+0000 7f00753ab640 1 -- 172.21.15.187:0/1257013839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f007013ed40 msgr2=0x7f00701415c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:11.040 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.036+0000 7f00753ab640 1 --2- 172.21.15.187:0/1257013839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f007013ed40 0x7f00701415c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f00600097a0 tx=0x7f006002b8d0 comp rx=0 tx=0).stop 2024-05-08T04:08:11.040 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.036+0000 7f00753ab640 1 -- 172.21.15.187:0/1257013839 shutdown_connections 2024-05-08T04:08:11.040 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.036+0000 7f00753ab640 1 --2- 172.21.15.187:0/1257013839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f007013ed40 0x7f00701415c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:11.041 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.036+0000 7f00753ab640 1 -- 172.21.15.187:0/1257013839 >> 172.21.15.187:0/1257013839 conn(0x7f00700b6020 msgr2=0x7f00700b6430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:11.041 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.036+0000 7f00753ab640 1 -- 172.21.15.187:0/1257013839 shutdown_connections 2024-05-08T04:08:11.041 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.036+0000 7f00753ab640 1 -- 172.21.15.187:0/1257013839 wait complete. 2024-05-08T04:08:11.041 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.037+0000 7f00753ab640 1 Processor -- start 2024-05-08T04:08:11.041 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.037+0000 7f00753ab640 1 -- start start 2024-05-08T04:08:11.041 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-08T04:08:11.038+0000 7f80f120d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T04:08:11.044 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.038+0000 7f00753ab640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0070075ea0 0x7f00700af200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:11.044 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.038+0000 7f00753ab640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00700b1090 0x7f007013d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:11.044 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.038+0000 7f00753ab640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f007013ed40 0x7f00701dc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:11.044 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.038+0000 7f00753ab640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f00700b16a0 con 0x7f007013ed40 2024-05-08T04:08:11.044 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.038+0000 7f00753ab640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f007014d0f0 con 0x7f00700b1090 2024-05-08T04:08:11.044 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.038+0000 7f00753ab640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f007014d270 con 0x7f0070075ea0 2024-05-08T04:08:11.044 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.038+0000 7f006e7fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00700b1090 0x7f007013d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:11.044 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.038+0000 7f006e7fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00700b1090 0x7f007013d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:44698/0 (socket says 172.21.15.187:44698) 2024-05-08T04:08:11.044 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.038+0000 7f006e7fc640 1 -- 172.21.15.187:0/3289887759 learned_addr learned my addr 172.21.15.187:0/3289887759 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-05-08T04:08:11.044 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.038+0000 7f006effd640 1 --2- 172.21.15.187:0/3289887759 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0070075ea0 0x7f00700af200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:11.044 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.038+0000 7f006f7fe640 1 --2- 172.21.15.187:0/3289887759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f007013ed40 0x7f00701dc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:11.045 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.038+0000 7f006e7fc640 1 -- 172.21.15.187:0/3289887759 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0070075ea0 msgr2=0x7f00700af200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:11.045 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.038+0000 7f006e7fc640 1 --2- 172.21.15.187:0/3289887759 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0070075ea0 0x7f00700af200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:11.045 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.038+0000 7f006e7fc640 1 -- 172.21.15.187:0/3289887759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f007013ed40 msgr2=0x7f00701dc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:11.045 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.038+0000 7f006e7fc640 1 --2- 172.21.15.187:0/3289887759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f007013ed40 0x7f00701dc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:11.045 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.038+0000 7f006e7fc640 1 -- 172.21.15.187:0/3289887759 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0070070c20 con 0x7f00700b1090 2024-05-08T04:08:11.045 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.038+0000 7f006e7fc640 1 --2- 172.21.15.187:0/3289887759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00700b1090 0x7f007013d540 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0060034040 tx=0x7f00600046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:11.045 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.039+0000 7f00577fe640 1 -- 172.21.15.187:0/3289887759 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0060040070 con 0x7f00700b1090 2024-05-08T04:08:11.045 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.039+0000 7f00577fe640 1 -- 172.21.15.187:0/3289887759 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f006003a830 con 0x7f00700b1090 2024-05-08T04:08:11.045 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.039+0000 7f00577fe640 1 -- 172.21.15.187:0/3289887759 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0060035040 con 0x7f00700b1090 2024-05-08T04:08:11.045 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.039+0000 7f00753ab640 1 -- 172.21.15.187:0/3289887759 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f007014cce0 con 0x7f00700b1090 2024-05-08T04:08:11.045 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.039+0000 7f00753ab640 1 -- 172.21.15.187:0/3289887759 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f007013cc60 con 0x7f00700b1090 2024-05-08T04:08:11.045 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.039+0000 7f00577fe640 1 -- 172.21.15.187:0/3289887759 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0060038020 con 0x7f00700b1090 2024-05-08T04:08:11.046 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.039+0000 7f00577fe640 1 -- 172.21.15.187:0/3289887759 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f006004dd50 con 0x7f00700b1090 2024-05-08T04:08:11.046 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.040+0000 7f00753ab640 1 -- 172.21.15.187:0/3289887759 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0070069bd0 con 0x7f00700b1090 2024-05-08T04:08:11.046 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.041+0000 7f00577fe640 1 -- 172.21.15.187:0/3289887759 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f007014cce0 con 0x7f00700b1090 2024-05-08T04:08:11.089 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-08T04:08:11.087+0000 7f80f120d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T04:08:11.115 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.113+0000 7f00753ab640 1 -- 172.21.15.187:0/3289887759 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f88f88db-216e-4acc-ba5a-89d91b98aeb4", "id": 5} v 0) -- 0x7f0070073d20 con 0x7f00700b1090 2024-05-08T04:08:11.123 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.121+0000 7f00577fe640 1 -- 172.21.15.187:0/3289887759 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f88f88db-216e-4acc-ba5a-89d91b98aeb4", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f006004d440 con 0x7f00700b1090 2024-05-08T04:08:11.123 INFO:teuthology.orchestra.run.smithi187.stdout:5 2024-05-08T04:08:11.124 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.122+0000 7f00753ab640 1 -- 172.21.15.187:0/3289887759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00700b1090 msgr2=0x7f007013d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:11.124 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.122+0000 7f00753ab640 1 --2- 172.21.15.187:0/3289887759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00700b1090 0x7f007013d540 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0060034040 tx=0x7f00600046c0 comp rx=0 tx=0).stop 2024-05-08T04:08:11.124 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.122+0000 7f00753ab640 1 -- 172.21.15.187:0/3289887759 shutdown_connections 2024-05-08T04:08:11.124 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.122+0000 7f00753ab640 1 --2- 172.21.15.187:0/3289887759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00700b1090 0x7f007013d540 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:11.124 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.122+0000 7f00753ab640 1 --2- 172.21.15.187:0/3289887759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f007013ed40 0x7f00701dc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:11.125 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.122+0000 7f00753ab640 1 --2- 172.21.15.187:0/3289887759 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0070075ea0 0x7f00700af200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:11.125 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.122+0000 7f00753ab640 1 -- 172.21.15.187:0/3289887759 >> 172.21.15.187:0/3289887759 conn(0x7f00700b6020 msgr2=0x7f0070141ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:11.125 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.122+0000 7f00753ab640 1 -- 172.21.15.187:0/3289887759 shutdown_connections 2024-05-08T04:08:11.125 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.122+0000 7f00753ab640 1 -- 172.21.15.187:0/3289887759 wait complete. 2024-05-08T04:08:11.131 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-05-08T04:08:11.130+0000 7f7ca884b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T04:08:11.134 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 6b6840fc-1e34-4cc8-9da6-8a57e69f319b 6 2024-05-08T04:08:11.144 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-08T04:08:11.142+0000 7f80f120d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T04:08:11.185 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-05-08T04:08:11.183+0000 7f7ca884b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T04:08:11.233 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.231+0000 7fcf3392b640 1 Processor -- start 2024-05-08T04:08:11.234 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.231+0000 7fcf3392b640 1 -- start start 2024-05-08T04:08:11.237 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.232+0000 7fcf3392b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf2c0724c0 0x7fcf2c145140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:11.237 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.232+0000 7fcf3392b640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf2c074150 con 0x7fcf2c13ef70 2024-05-08T04:08:11.237 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.232+0000 7fcf3392b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf2c074430 con 0x7fcf2c0b1980 2024-05-08T04:08:11.237 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.232+0000 7fcf3392b640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fcf2c144cb0 con 0x7fcf2c0724c0 2024-05-08T04:08:11.237 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.232+0000 7fcf30ec6640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fcf2c0b1980 0x7fcf2c13e860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.187:40900/0 (socket says 172.21.15.187:40900) 2024-05-08T04:08:11.237 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.232+0000 7fcf30ec6640 1 -- 172.21.15.187:0/778477409 learned_addr learned my addr 172.21.15.187:0/778477409 (peer_addr_for_me v1:172.21.15.187:0/0) 2024-05-08T04:08:11.237 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.232+0000 7fcf316c7640 1 --2- 172.21.15.187:0/778477409 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf2c0724c0 0x7fcf2c145140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:11.237 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.232+0000 7fcf1ffff640 1 -- 172.21.15.187:0/778477409 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3218981557 0 0) 0x7fcf2c074430 con 0x7fcf2c0b1980 2024-05-08T04:08:11.237 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.232+0000 7fcf1ffff640 1 -- 172.21.15.187:0/778477409 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf18003520 con 0x7fcf2c0b1980 2024-05-08T04:08:11.237 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.233+0000 7fcf1ffff640 1 -- 172.21.15.187:0/778477409 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1660005166 0 0) 0x7fcf2c074150 con 0x7fcf2c13ef70 2024-05-08T04:08:11.237 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.233+0000 7fcf1ffff640 1 -- 172.21.15.187:0/778477409 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf2c074430 con 0x7fcf2c13ef70 2024-05-08T04:08:11.238 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.233+0000 7fcf316c7640 1 -- 172.21.15.187:0/778477409 >> v1:172.21.15.187:6789/0 conn(0x7fcf2c0b1980 legacy=0x7fcf2c13e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:11.238 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.233+0000 7fcf316c7640 1 -- 172.21.15.187:0/778477409 >> v1:172.21.15.170:6789/0 conn(0x7fcf2c13ef70 legacy=0x7fcf2c14be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:11.238 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.233+0000 7fcf316c7640 1 -- 172.21.15.187:0/778477409 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf2c0aa650 con 0x7fcf2c0724c0 2024-05-08T04:08:11.238 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.233+0000 7fcf316c7640 1 --2- 172.21.15.187:0/778477409 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf2c0724c0 0x7fcf2c145140 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcf280097d0 tx=0x7fcf2802b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69c0167edd0b326e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:11.238 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.234+0000 7fcf1ffff640 1 -- 172.21.15.187:0/778477409 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf28031770 con 0x7fcf2c0724c0 2024-05-08T04:08:11.238 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.234+0000 7fcf1ffff640 1 -- 172.21.15.187:0/778477409 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf28037070 con 0x7fcf2c0724c0 2024-05-08T04:08:11.238 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.234+0000 7fcf1ffff640 1 -- 172.21.15.187:0/778477409 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf28031b10 con 0x7fcf2c0724c0 2024-05-08T04:08:11.238 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.234+0000 7fcf3392b640 1 -- 172.21.15.187:0/778477409 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf2c0724c0 msgr2=0x7fcf2c145140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:11.238 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.234+0000 7fcf3392b640 1 --2- 172.21.15.187:0/778477409 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf2c0724c0 0x7fcf2c145140 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcf280097d0 tx=0x7fcf2802b810 comp rx=0 tx=0).stop 2024-05-08T04:08:11.238 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.234+0000 7fcf3392b640 1 -- 172.21.15.187:0/778477409 shutdown_connections 2024-05-08T04:08:11.238 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.234+0000 7fcf3392b640 1 --2- 172.21.15.187:0/778477409 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf2c0724c0 0x7fcf2c145140 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:11.238 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.234+0000 7fcf3392b640 1 -- 172.21.15.187:0/778477409 >> 172.21.15.187:0/778477409 conn(0x7fcf2c0b7110 msgr2=0x7fcf2c0b7520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:11.238 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.234+0000 7fcf3392b640 1 -- 172.21.15.187:0/778477409 shutdown_connections 2024-05-08T04:08:11.238 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.234+0000 7fcf3392b640 1 -- 172.21.15.187:0/778477409 wait complete. 2024-05-08T04:08:11.239 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-05-08T04:08:11.236+0000 7f7ca884b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T04:08:11.241 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.235+0000 7fcf3392b640 1 Processor -- start 2024-05-08T04:08:11.242 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.235+0000 7fcf3392b640 1 -- start start 2024-05-08T04:08:11.242 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.236+0000 7fcf3392b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf2c0724c0 0x7fcf2c0ab090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:11.242 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.236+0000 7fcf3392b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf2c0b1980 0x7fcf2c1d6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:11.242 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.236+0000 7fcf3392b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf2c13ef70 0x7fcf2c1d6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:11.242 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.236+0000 7fcf3392b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c145680 con 0x7fcf2c0b1980 2024-05-08T04:08:11.242 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.236+0000 7fcf3392b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c072ab0 con 0x7fcf2c13ef70 2024-05-08T04:08:11.242 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.236+0000 7fcf3392b640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fcf2c145800 con 0x7fcf2c0724c0 2024-05-08T04:08:11.242 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.236+0000 7fcf316c7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf2c0724c0 0x7fcf2c0ab090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:11.242 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.236+0000 7fcf31ec8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf2c13ef70 0x7fcf2c1d6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:11.242 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.236+0000 7fcf316c7640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf2c0724c0 0x7fcf2c0ab090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.187:37744/0 (socket says 172.21.15.187:37744) 2024-05-08T04:08:11.242 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.236+0000 7fcf30ec6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf2c0b1980 0x7fcf2c1d6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:11.242 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.236+0000 7fcf316c7640 1 -- 172.21.15.187:0/246788450 learned_addr learned my addr 172.21.15.187:0/246788450 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-05-08T04:08:11.242 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.236+0000 7fcf30ec6640 1 -- 172.21.15.187:0/246788450 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf2c0724c0 msgr2=0x7fcf2c0ab090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:11.242 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.236+0000 7fcf30ec6640 1 --2- 172.21.15.187:0/246788450 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf2c0724c0 0x7fcf2c0ab090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:11.243 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.236+0000 7fcf30ec6640 1 -- 172.21.15.187:0/246788450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf2c13ef70 msgr2=0x7fcf2c1d6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:11.243 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.236+0000 7fcf30ec6640 1 --2- 172.21.15.187:0/246788450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf2c13ef70 0x7fcf2c1d6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:11.243 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.236+0000 7fcf30ec6640 1 -- 172.21.15.187:0/246788450 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf2c0acec0 con 0x7fcf2c0b1980 2024-05-08T04:08:11.243 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.236+0000 7fcf316c7640 1 --2- 172.21.15.187:0/246788450 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf2c0724c0 0x7fcf2c0ab090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:08:11.243 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.237+0000 7fcf30ec6640 1 --2- 172.21.15.187:0/246788450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf2c0b1980 0x7fcf2c1d6920 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcf2000c430 tx=0x7fcf2000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:11.243 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.237+0000 7fcf1dffb640 1 -- 172.21.15.187:0/246788450 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf20017070 con 0x7fcf2c0b1980 2024-05-08T04:08:11.243 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.237+0000 7fcf1dffb640 1 -- 172.21.15.187:0/246788450 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf20005ce0 con 0x7fcf2c0b1980 2024-05-08T04:08:11.243 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.237+0000 7fcf3392b640 1 -- 172.21.15.187:0/246788450 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf2c0ad200 con 0x7fcf2c0b1980 2024-05-08T04:08:11.243 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.237+0000 7fcf1dffb640 1 -- 172.21.15.187:0/246788450 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf200072d0 con 0x7fcf2c0b1980 2024-05-08T04:08:11.243 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.237+0000 7fcf3392b640 1 -- 172.21.15.187:0/246788450 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf2c1d7580 con 0x7fcf2c0b1980 2024-05-08T04:08:11.243 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.237+0000 7fcf1dffb640 1 -- 172.21.15.187:0/246788450 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcf2001a070 con 0x7fcf2c0b1980 2024-05-08T04:08:11.243 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.237+0000 7fcf1dffb640 1 -- 172.21.15.187:0/246788450 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fcf2001cbc0 con 0x7fcf2c0b1980 2024-05-08T04:08:11.243 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.238+0000 7fcf3392b640 1 -- 172.21.15.187:0/246788450 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf2c069bd0 con 0x7fcf2c0b1980 2024-05-08T04:08:11.244 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.240+0000 7fcf1dffb640 1 -- 172.21.15.187:0/246788450 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fcf2c0ad200 con 0x7fcf2c0b1980 2024-05-08T04:08:11.263 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-08T04:08:11.261+0000 7f80f120d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T04:08:11.314 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.312+0000 7fcf3392b640 1 -- 172.21.15.187:0/246788450 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6b6840fc-1e34-4cc8-9da6-8a57e69f319b", "id": 6} v 0) -- 0x7fcf2c06e970 con 0x7fcf2c0b1980 2024-05-08T04:08:11.332 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.313+0000 7fcf1dffb640 1 -- 172.21.15.187:0/246788450 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6b6840fc-1e34-4cc8-9da6-8a57e69f319b", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fcf20007540 con 0x7fcf2c0b1980 2024-05-08T04:08:11.332 INFO:teuthology.orchestra.run.smithi187.stdout:6 2024-05-08T04:08:11.334 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.314+0000 7fcf3392b640 1 -- 172.21.15.187:0/246788450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf2c0b1980 msgr2=0x7fcf2c1d6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:11.334 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.314+0000 7fcf3392b640 1 --2- 172.21.15.187:0/246788450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf2c0b1980 0x7fcf2c1d6920 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcf2000c430 tx=0x7fcf2000c8f0 comp rx=0 tx=0).stop 2024-05-08T04:08:11.334 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.315+0000 7fcf3392b640 1 -- 172.21.15.187:0/246788450 shutdown_connections 2024-05-08T04:08:11.334 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.315+0000 7fcf3392b640 1 --2- 172.21.15.187:0/246788450 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcf2c13ef70 0x7fcf2c1d6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:11.334 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.315+0000 7fcf3392b640 1 --2- 172.21.15.187:0/246788450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf2c0b1980 0x7fcf2c1d6920 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:11.334 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.315+0000 7fcf3392b640 1 --2- 172.21.15.187:0/246788450 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf2c0724c0 0x7fcf2c0ab090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:11.334 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.315+0000 7fcf3392b640 1 -- 172.21.15.187:0/246788450 >> 172.21.15.187:0/246788450 conn(0x7fcf2c0b7110 msgr2=0x7fcf2c0b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:11.334 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.315+0000 7fcf3392b640 1 -- 172.21.15.187:0/246788450 shutdown_connections 2024-05-08T04:08:11.334 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.315+0000 7fcf3392b640 1 -- 172.21.15.187:0/246788450 wait complete. 2024-05-08T04:08:11.335 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 5dbcf804-181b-4d5b-9a40-f99ddaee2701 7 2024-05-08T04:08:11.353 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-05-08T04:08:11.351+0000 7f7ca884b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T04:08:11.473 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.471+0000 7fdd9a427640 1 Processor -- start 2024-05-08T04:08:11.473 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.471+0000 7fdd9a427640 1 -- start start 2024-05-08T04:08:11.475 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.472+0000 7fdd9a427640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd9406cf00 0x7fdd941461d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:11.475 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.472+0000 7fdd9a427640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fdd94073320 0x7fdd9414e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:11.475 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.472+0000 7fdd9a427640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd94072510 con 0x7fdd94146710 2024-05-08T04:08:11.475 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.472+0000 7fdd9a427640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7fdd940727f0 con 0x7fdd94073320 2024-05-08T04:08:11.476 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.472+0000 7fdd9a427640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fdd9406dd30 con 0x7fdd9406cf00 2024-05-08T04:08:11.476 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.472+0000 7fdd8bfff640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fdd94146710 0x7fdd94072c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.187:40906/0 (socket says 172.21.15.187:40906) 2024-05-08T04:08:11.476 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.472+0000 7fdd8bfff640 1 -- 172.21.15.187:0/1880988690 learned_addr learned my addr 172.21.15.187:0/1880988690 (peer_addr_for_me v1:172.21.15.187:0/0) 2024-05-08T04:08:11.476 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.472+0000 7fdd93fff640 1 --2- 172.21.15.187:0/1880988690 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd9406cf00 0x7fdd941461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:11.476 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.472+0000 7fdd989c4640 1 --2- 172.21.15.187:0/1880988690 >> v2:172.21.15.170:3300/0 conn(0x7fdd94073320 0x7fdd9414e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:11.476 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.472+0000 7fdd937fe640 1 -- 172.21.15.187:0/1880988690 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2339526533 0 0) 0x7fdd94072510 con 0x7fdd94146710 2024-05-08T04:08:11.476 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.472+0000 7fdd937fe640 1 -- 172.21.15.187:0/1880988690 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd80003520 con 0x7fdd94146710 2024-05-08T04:08:11.476 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.472+0000 7fdd93fff640 1 -- 172.21.15.187:0/1880988690 >> v2:172.21.15.170:3300/0 conn(0x7fdd94073320 msgr2=0x7fdd9414e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:11.476 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.472+0000 7fdd93fff640 1 --2- 172.21.15.187:0/1880988690 >> v2:172.21.15.170:3300/0 conn(0x7fdd94073320 0x7fdd9414e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:11.476 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.472+0000 7fdd93fff640 1 -- 172.21.15.187:0/1880988690 >> v1:172.21.15.187:6789/0 conn(0x7fdd94146710 legacy=0x7fdd94072c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:11.476 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.472+0000 7fdd93fff640 1 -- 172.21.15.187:0/1880988690 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd94146b50 con 0x7fdd9406cf00 2024-05-08T04:08:11.476 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.472+0000 7fdd989c4640 1 --2- 172.21.15.187:0/1880988690 >> v2:172.21.15.170:3300/0 conn(0x7fdd94073320 0x7fdd9414e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:08:11.476 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.473+0000 7fdd93fff640 1 --2- 172.21.15.187:0/1880988690 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd9406cf00 0x7fdd941461d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdd840097a0 tx=0x7fdd8402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ad092e01b73c3f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:11.476 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.473+0000 7fdd937fe640 1 -- 172.21.15.187:0/1880988690 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd840315a0 con 0x7fdd9406cf00 2024-05-08T04:08:11.476 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.473+0000 7fdd937fe640 1 -- 172.21.15.187:0/1880988690 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd84037070 con 0x7fdd9406cf00 2024-05-08T04:08:11.476 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.473+0000 7fdd937fe640 1 -- 172.21.15.187:0/1880988690 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd84031960 con 0x7fdd9406cf00 2024-05-08T04:08:11.479 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.474+0000 7fdd9a427640 1 -- 172.21.15.187:0/1880988690 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd9406cf00 msgr2=0x7fdd941461d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:11.479 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.474+0000 7fdd9a427640 1 --2- 172.21.15.187:0/1880988690 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd9406cf00 0x7fdd941461d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdd840097a0 tx=0x7fdd8402b840 comp rx=0 tx=0).stop 2024-05-08T04:08:11.479 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.474+0000 7fdd9a427640 1 -- 172.21.15.187:0/1880988690 shutdown_connections 2024-05-08T04:08:11.479 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.474+0000 7fdd9a427640 1 --2- 172.21.15.187:0/1880988690 >> v2:172.21.15.170:3300/0 conn(0x7fdd94073320 0x7fdd9414e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:11.479 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.474+0000 7fdd9a427640 1 --2- 172.21.15.187:0/1880988690 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd9406cf00 0x7fdd941461d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:11.479 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.474+0000 7fdd9a427640 1 -- 172.21.15.187:0/1880988690 >> 172.21.15.187:0/1880988690 conn(0x7fdd940b7110 msgr2=0x7fdd940b7520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:11.480 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.474+0000 7fdd9a427640 1 -- 172.21.15.187:0/1880988690 shutdown_connections 2024-05-08T04:08:11.480 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.474+0000 7fdd9a427640 1 -- 172.21.15.187:0/1880988690 wait complete. 2024-05-08T04:08:11.480 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.475+0000 7fdd9a427640 1 Processor -- start 2024-05-08T04:08:11.480 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.475+0000 7fdd9a427640 1 -- start start 2024-05-08T04:08:11.480 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.476+0000 7fdd9a427640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd9406cf00 0x7fdd941241e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:11.480 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.476+0000 7fdd9a427640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd94073320 0x7fdd94143090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:11.480 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.476+0000 7fdd9a427640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd94146710 0x7fdd941435d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:11.480 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.476+0000 7fdd9a427640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdd94073930 con 0x7fdd94073320 2024-05-08T04:08:11.480 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.476+0000 7fdd9a427640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdd940737b0 con 0x7fdd9406cf00 2024-05-08T04:08:11.480 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.476+0000 7fdd9a427640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fdd9406d2e0 con 0x7fdd94146710 2024-05-08T04:08:11.480 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.476+0000 7fdd93fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd9406cf00 0x7fdd941241e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:11.480 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.476+0000 7fdd93fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd9406cf00 0x7fdd941241e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:44728/0 (socket says 172.21.15.187:44728) 2024-05-08T04:08:11.481 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.476+0000 7fdd93fff640 1 -- 172.21.15.187:0/4049826691 learned_addr learned my addr 172.21.15.187:0/4049826691 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-05-08T04:08:11.481 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.476+0000 7fdd989c4640 1 --2- 172.21.15.187:0/4049826691 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd94146710 0x7fdd941435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:11.481 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.476+0000 7fdd8bfff640 1 --2- 172.21.15.187:0/4049826691 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd94073320 0x7fdd94143090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:11.481 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.476+0000 7fdd93fff640 1 -- 172.21.15.187:0/4049826691 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd94146710 msgr2=0x7fdd941435d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:11.481 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.476+0000 7fdd93fff640 1 --2- 172.21.15.187:0/4049826691 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd94146710 0x7fdd941435d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:11.481 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.476+0000 7fdd93fff640 1 -- 172.21.15.187:0/4049826691 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd94073320 msgr2=0x7fdd94143090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:11.481 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.476+0000 7fdd93fff640 1 --2- 172.21.15.187:0/4049826691 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd94073320 0x7fdd94143090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:11.481 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.476+0000 7fdd93fff640 1 -- 172.21.15.187:0/4049826691 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd941265d0 con 0x7fdd9406cf00 2024-05-08T04:08:11.481 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.476+0000 7fdd93fff640 1 --2- 172.21.15.187:0/4049826691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd9406cf00 0x7fdd941241e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdd840097a0 tx=0x7fdd840046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:11.481 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.476+0000 7fdd917fa640 1 -- 172.21.15.187:0/4049826691 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd84032630 con 0x7fdd9406cf00 2024-05-08T04:08:11.481 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.476+0000 7fdd917fa640 1 -- 172.21.15.187:0/4049826691 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd840327d0 con 0x7fdd9406cf00 2024-05-08T04:08:11.481 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.476+0000 7fdd9a427640 1 -- 172.21.15.187:0/4049826691 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd94143b10 con 0x7fdd9406cf00 2024-05-08T04:08:11.481 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.476+0000 7fdd917fa640 1 -- 172.21.15.187:0/4049826691 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd84032bc0 con 0x7fdd9406cf00 2024-05-08T04:08:11.481 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.476+0000 7fdd9a427640 1 -- 172.21.15.187:0/4049826691 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd94124f80 con 0x7fdd9406cf00 2024-05-08T04:08:11.482 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.477+0000 7fdd917fa640 1 -- 172.21.15.187:0/4049826691 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdd84006020 con 0x7fdd9406cf00 2024-05-08T04:08:11.482 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.477+0000 7fdd917fa640 1 -- 172.21.15.187:0/4049826691 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fdd8402f030 con 0x7fdd9406cf00 2024-05-08T04:08:11.482 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.478+0000 7fdd9a427640 1 -- 172.21.15.187:0/4049826691 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd94069bd0 con 0x7fdd9406cf00 2024-05-08T04:08:11.482 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.480+0000 7fdd917fa640 1 -- 172.21.15.187:0/4049826691 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fdd84040070 con 0x7fdd9406cf00 2024-05-08T04:08:11.562 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.560+0000 7fdd9a427640 1 -- 172.21.15.187:0/4049826691 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5dbcf804-181b-4d5b-9a40-f99ddaee2701", "id": 7} v 0) -- 0x7fdd9406e970 con 0x7fdd9406cf00 2024-05-08T04:08:11.565 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.563+0000 7fdd917fa640 1 -- 172.21.15.187:0/4049826691 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5dbcf804-181b-4d5b-9a40-f99ddaee2701", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fdd84005e10 con 0x7fdd9406cf00 2024-05-08T04:08:11.565 INFO:teuthology.orchestra.run.smithi187.stdout:7 2024-05-08T04:08:11.567 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.564+0000 7fdd9a427640 1 -- 172.21.15.187:0/4049826691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd9406cf00 msgr2=0x7fdd941241e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:11.567 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.564+0000 7fdd9a427640 1 --2- 172.21.15.187:0/4049826691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd9406cf00 0x7fdd941241e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdd840097a0 tx=0x7fdd840046c0 comp rx=0 tx=0).stop 2024-05-08T04:08:11.567 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.564+0000 7fdd9a427640 1 -- 172.21.15.187:0/4049826691 shutdown_connections 2024-05-08T04:08:11.567 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.564+0000 7fdd9a427640 1 --2- 172.21.15.187:0/4049826691 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd94146710 0x7fdd941435d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:11.567 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.564+0000 7fdd9a427640 1 --2- 172.21.15.187:0/4049826691 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd94073320 0x7fdd94143090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:11.567 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.564+0000 7fdd9a427640 1 --2- 172.21.15.187:0/4049826691 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd9406cf00 0x7fdd941241e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:11.567 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.564+0000 7fdd9a427640 1 -- 172.21.15.187:0/4049826691 >> 172.21.15.187:0/4049826691 conn(0x7fdd940b7110 msgr2=0x7fdd940b1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:11.567 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.564+0000 7fdd9a427640 1 -- 172.21.15.187:0/4049826691 shutdown_connections 2024-05-08T04:08:11.567 INFO:teuthology.orchestra.run.smithi187.stderr:2024-05-08T04:08:11.564+0000 7fdd9a427640 1 -- 172.21.15.187:0/4049826691 wait complete. 2024-05-08T04:08:11.577 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-08T04:08:11.577 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-08T04:08:11.579 INFO:tasks.ceph.osd.0:Started 2024-05-08T04:08:11.579 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-08T04:08:11.579 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-08T04:08:11.581 INFO:tasks.ceph.osd.1:Started 2024-05-08T04:08:11.582 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-08T04:08:11.582 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-08T04:08:11.584 INFO:tasks.ceph.osd.2:Started 2024-05-08T04:08:11.584 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-08T04:08:11.584 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-08T04:08:11.586 INFO:tasks.ceph.osd.3:Started 2024-05-08T04:08:11.586 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-08T04:08:11.586 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-08T04:08:11.589 INFO:tasks.ceph.osd.4:Started 2024-05-08T04:08:11.589 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-08T04:08:11.589 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-08T04:08:11.591 INFO:tasks.ceph.osd.5:Started 2024-05-08T04:08:11.591 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-08T04:08:11.591 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-08T04:08:11.593 INFO:tasks.ceph.osd.6:Started 2024-05-08T04:08:11.593 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-08T04:08:11.593 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-08T04:08:11.596 INFO:tasks.ceph.osd.7:Started 2024-05-08T04:08:11.596 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:08:11.680 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-08T04:08:11.678+0000 7f80f120d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T04:08:11.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.713+0000 7f2ad6ffd640 1 Processor -- start 2024-05-08T04:08:11.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.713+0000 7f2ad6ffd640 1 -- start start 2024-05-08T04:08:11.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.714+0000 7f2ad6ffd640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2ad00b0890 0x7f2ad01410c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:11.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.714+0000 7f2ad6ffd640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ad00ae8c0 con 0x7f2ad006d950 2024-05-08T04:08:11.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.714+0000 7f2ad6ffd640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ad0146dd0 con 0x7f2ad0140c50 2024-05-08T04:08:11.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.714+0000 7f2ad6ffd640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f2ad00af760 con 0x7f2ad00b0890 2024-05-08T04:08:11.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.714+0000 7f2ad4d99640 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f2ad006d950 0x7f2ad006dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:36626/0 (socket says 172.21.15.170:36626) 2024-05-08T04:08:11.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.714+0000 7f2ad4d99640 1 -- 172.21.15.170:0/3324359348 learned_addr learned my addr 172.21.15.170:0/3324359348 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-05-08T04:08:11.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.714+0000 7f2ad559a640 1 --2- 172.21.15.170:0/3324359348 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2ad00b0890 0x7f2ad01410c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:11.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.714+0000 7f2ad559a640 1 -- 172.21.15.170:0/3324359348 >> v1:172.21.15.187:6789/0 conn(0x7f2ad0140c50 legacy=0x7f2ad00b0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:11.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.714+0000 7f2ad559a640 1 -- 172.21.15.170:0/3324359348 >> v1:172.21.15.170:6789/0 conn(0x7f2ad006d950 legacy=0x7f2ad006dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:11.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.715+0000 7f2ad559a640 1 -- 172.21.15.170:0/3324359348 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad006d0e0 con 0x7f2ad00b0890 2024-05-08T04:08:11.717 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.715+0000 7f2ad559a640 1 --2- 172.21.15.170:0/3324359348 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2ad00b0890 0x7f2ad01410c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8009dc0 tx=0x7f2ac802bcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e6575ecd4ae16fe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:11.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.715+0000 7f2acf7fe640 1 -- 172.21.15.170:0/3324359348 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8039070 con 0x7f2ad00b0890 2024-05-08T04:08:11.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.715+0000 7f2acf7fe640 1 -- 172.21.15.170:0/3324359348 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ac802c9f0 con 0x7f2ad00b0890 2024-05-08T04:08:11.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.715+0000 7f2acf7fe640 1 -- 172.21.15.170:0/3324359348 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac802cdc0 con 0x7f2ad00b0890 2024-05-08T04:08:11.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.716+0000 7f2ad6ffd640 1 -- 172.21.15.170:0/3324359348 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2ad00b0890 msgr2=0x7f2ad01410c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:11.718 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.716+0000 7f2ad6ffd640 1 --2- 172.21.15.170:0/3324359348 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2ad00b0890 0x7f2ad01410c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8009dc0 tx=0x7f2ac802bcc0 comp rx=0 tx=0).stop 2024-05-08T04:08:11.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.716+0000 7f2ad6ffd640 1 -- 172.21.15.170:0/3324359348 shutdown_connections 2024-05-08T04:08:11.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.716+0000 7f2ad6ffd640 1 --2- 172.21.15.170:0/3324359348 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2ad00b0890 0x7f2ad01410c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:11.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.716+0000 7f2ad6ffd640 1 -- 172.21.15.170:0/3324359348 >> 172.21.15.170:0/3324359348 conn(0x7f2ad00b7270 msgr2=0x7f2ad00b7680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:11.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.716+0000 7f2ad6ffd640 1 -- 172.21.15.170:0/3324359348 shutdown_connections 2024-05-08T04:08:11.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.716+0000 7f2ad6ffd640 1 -- 172.21.15.170:0/3324359348 wait complete. 2024-05-08T04:08:11.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.717+0000 7f2ad6ffd640 1 Processor -- start 2024-05-08T04:08:11.719 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.717+0000 7f2ad6ffd640 1 -- start start 2024-05-08T04:08:11.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.718+0000 7f2ad6ffd640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2ad006d950 0x7f2ad0139710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:11.721 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.718+0000 7f2ad6ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ad00b0890 0x7f2ad0132ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:11.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.718+0000 7f2ad6ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ad0140c50 0x7f2ad0133000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:11.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.718+0000 7f2ad6ffd640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad014f4e0 con 0x7f2ad0140c50 2024-05-08T04:08:11.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.718+0000 7f2ad6ffd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad014f360 con 0x7f2ad00b0890 2024-05-08T04:08:11.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.718+0000 7f2ad559a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ad0140c50 0x7f2ad0133000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:11.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.718+0000 7f2ad4d99640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2ad006d950 0x7f2ad0139710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:11.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.718+0000 7f2ad559a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ad0140c50 0x7f2ad0133000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48480/0 (socket says 172.21.15.170:48480) 2024-05-08T04:08:11.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.718+0000 7f2ad559a640 1 -- 172.21.15.170:0/1559932014 learned_addr learned my addr 172.21.15.170:0/1559932014 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:11.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.718+0000 7f2ad6ffd640 1 -- 172.21.15.170:0/1559932014 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f2ad014f660 con 0x7f2ad006d950 2024-05-08T04:08:11.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.718+0000 7f2acffff640 1 --2- 172.21.15.170:0/1559932014 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ad00b0890 0x7f2ad0132ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:11.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.719+0000 7f2ad559a640 1 -- 172.21.15.170:0/1559932014 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2ad006d950 msgr2=0x7f2ad0139710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:11.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.719+0000 7f2ad559a640 1 --2- 172.21.15.170:0/1559932014 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2ad006d950 0x7f2ad0139710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:11.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.719+0000 7f2ad559a640 1 -- 172.21.15.170:0/1559932014 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ad00b0890 msgr2=0x7f2ad0132ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:11.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.719+0000 7f2ad559a640 1 --2- 172.21.15.170:0/1559932014 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ad00b0890 0x7f2ad0132ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:11.722 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.719+0000 7f2ad559a640 1 -- 172.21.15.170:0/1559932014 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad0141690 con 0x7f2ad0140c50 2024-05-08T04:08:11.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.719+0000 7f2ad559a640 1 --2- 172.21.15.170:0/1559932014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ad0140c50 0x7f2ad0133000 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8009dc0 tx=0x7f2ac8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:11.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.719+0000 7f2acd7fa640 1 -- 172.21.15.170:0/1559932014 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8039050 con 0x7f2ad0140c50 2024-05-08T04:08:11.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.719+0000 7f2acd7fa640 1 -- 172.21.15.170:0/1559932014 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ac8045070 con 0x7f2ad0140c50 2024-05-08T04:08:11.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.719+0000 7f2ad6ffd640 1 -- 172.21.15.170:0/1559932014 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ad013ad60 con 0x7f2ad0140c50 2024-05-08T04:08:11.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.719+0000 7f2acd7fa640 1 -- 172.21.15.170:0/1559932014 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8040690 con 0x7f2ad0140c50 2024-05-08T04:08:11.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.719+0000 7f2ad6ffd640 1 -- 172.21.15.170:0/1559932014 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ad013b320 con 0x7f2ad0140c50 2024-05-08T04:08:11.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.719+0000 7f2acd7fa640 1 -- 172.21.15.170:0/1559932014 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2ac80408b0 con 0x7f2ad0140c50 2024-05-08T04:08:11.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.720+0000 7f2acd7fa640 1 -- 172.21.15.170:0/1559932014 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f2ac8040d30 con 0x7f2ad0140c50 2024-05-08T04:08:11.723 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.720+0000 7f2ad6ffd640 1 -- 172.21.15.170:0/1559932014 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ad0069bd0 con 0x7f2ad0140c50 2024-05-08T04:08:11.724 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.722+0000 7f2acd7fa640 1 -- 172.21.15.170:0/1559932014 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f2ac804f020 con 0x7f2ad0140c50 2024-05-08T04:08:11.783 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-05-08T04:08:11.782+0000 7f7ca884b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T04:08:11.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.784+0000 7f2ad6ffd640 1 -- 172.21.15.170:0/1559932014 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ad006e7e0 con 0x7f2ad0140c50 2024-05-08T04:08:11.786 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.784+0000 7f2acd7fa640 1 -- 172.21.15.170:0/1559932014 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f2ac802c810 con 0x7f2ad0140c50 2024-05-08T04:08:11.786 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:08:11.786 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":10,"fsid":"e5497bb3-afa8-4f24-8ef4-42911ffcc962","created":"2024-05-08T04:08:09.270498+0000","modified":"2024-05-08T04:08:11.562526+0000","last_up_change":"0.000000","last_in_change":"2024-05-08T04:08:11.562526+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6bca968b-d825-417a-b907-5312aad8dbcd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"66600f89-4532-43b1-85d3-6d8c736cfaf2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a11608b2-0871-497c-aa38-264baea6bf39","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"8caae6f8-790c-426c-af8a-3a9ef54eae48","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0e804208-3607-4d71-b771-f79fb27e7968","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f88f88db-216e-4acc-ba5a-89d91b98aeb4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6b6840fc-1e34-4cc8-9da6-8a57e69f319b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5dbcf804-181b-4d5b-9a40-f99ddaee2701","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:08:11.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.785+0000 7f2ad6ffd640 1 -- 172.21.15.170:0/1559932014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ad0140c50 msgr2=0x7f2ad0133000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:11.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.785+0000 7f2ad6ffd640 1 --2- 172.21.15.170:0/1559932014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ad0140c50 0x7f2ad0133000 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8009dc0 tx=0x7f2ac8004a40 comp rx=0 tx=0).stop 2024-05-08T04:08:11.787 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.785+0000 7f2ad6ffd640 1 -- 172.21.15.170:0/1559932014 shutdown_connections 2024-05-08T04:08:11.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.785+0000 7f2ad6ffd640 1 --2- 172.21.15.170:0/1559932014 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2ad00b0890 0x7f2ad0132ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:11.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.785+0000 7f2ad6ffd640 1 --2- 172.21.15.170:0/1559932014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ad0140c50 0x7f2ad0133000 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:11.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.785+0000 7f2ad6ffd640 1 --2- 172.21.15.170:0/1559932014 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2ad006d950 0x7f2ad0139710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:11.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.785+0000 7f2ad6ffd640 1 -- 172.21.15.170:0/1559932014 >> 172.21.15.170:0/1559932014 conn(0x7f2ad00b7270 msgr2=0x7f2ad00b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:11.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.785+0000 7f2ad6ffd640 1 -- 172.21.15.170:0/1559932014 shutdown_connections 2024-05-08T04:08:11.788 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:11.785+0000 7f2ad6ffd640 1 -- 172.21.15.170:0/1559932014 wait complete. 2024-05-08T04:08:11.798 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-08T04:08:11.798 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-08T04:08:11.822 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-08T04:08:11.820+0000 7f80f120d040 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-08T04:08:11.925 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-05-08T04:08:11.923+0000 7f7ca884b040 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-08T04:08:12.084 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-08T04:08:12.082+0000 7f80f120d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T04:08:12.136 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-08T04:08:12.134+0000 7f80f120d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T04:08:12.186 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-05-08T04:08:12.184+0000 7f7ca884b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T04:08:12.191 INFO:tasks.ceph.osd.0.smithi170.stderr:2024-05-08T04:08:12.189+0000 7f8d9120f640 -1 Falling back to public interface 2024-05-08T04:08:12.194 INFO:tasks.ceph.osd.5.smithi187.stderr:2024-05-08T04:08:12.192+0000 7f6c785fc640 -1 Falling back to public interface 2024-05-08T04:08:12.199 INFO:tasks.ceph.osd.7.smithi187.stderr:2024-05-08T04:08:12.197+0000 7fc94bc1b640 -1 Falling back to public interface 2024-05-08T04:08:12.199 INFO:tasks.ceph.osd.2.smithi170.stderr:2024-05-08T04:08:12.197+0000 7f60a9013640 -1 Falling back to public interface 2024-05-08T04:08:12.208 INFO:tasks.ceph.osd.1.smithi170.stderr:2024-05-08T04:08:12.206+0000 7f7975ab1640 -1 Falling back to public interface 2024-05-08T04:08:12.211 INFO:tasks.ceph.osd.3.smithi170.stderr:2024-05-08T04:08:12.209+0000 7fd33f91a640 -1 Falling back to public interface 2024-05-08T04:08:12.216 INFO:tasks.ceph.osd.4.smithi187.stderr:2024-05-08T04:08:12.214+0000 7fba89d6e640 -1 Falling back to public interface 2024-05-08T04:08:12.217 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-05-08T04:08:12.214+0000 7fcec7e15640 -1 Falling back to public interface 2024-05-08T04:08:12.217 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-05-08T04:08:12.214+0000 7fcec6701640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6808/0: (98) Address already in use 2024-05-08T04:08:12.240 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-05-08T04:08:12.238+0000 7f7ca884b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T04:08:12.255 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-08T04:08:12.254+0000 7f80f120d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T04:08:12.359 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-05-08T04:08:12.357+0000 7f7ca884b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T04:08:12.546 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-08T04:08:12.544+0000 7f80f120d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T04:08:12.593 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-08T04:08:12.591+0000 7f80f120d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T04:08:12.602 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T04:08:12.663 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-05-08T04:08:12.661+0000 7f7ca884b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T04:08:12.694 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.692+0000 7f216b7c0640 1 Processor -- start 2024-05-08T04:08:12.695 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.693+0000 7f216b7c0640 1 -- start start 2024-05-08T04:08:12.696 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.693+0000 7f216b7c0640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2164075eb0 0x7f2164140990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:12.696 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.693+0000 7f216b7c0640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f21640b15d0 0x7f2164140ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:12.696 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.693+0000 7f216b7c0640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2164146e90 0x7f21641472c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:12.696 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.693+0000 7f216b7c0640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7f2164076290 con 0x7f2164075eb0 2024-05-08T04:08:12.696 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.693+0000 7f216b7c0640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f2164141410 con 0x7f21640b15d0 2024-05-08T04:08:12.696 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.693+0000 7f216b7c0640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f2164070690 con 0x7f2164146e90 2024-05-08T04:08:12.696 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.693+0000 7f2169d5d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2164146e90 0x7f21641472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:12.697 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.693+0000 7f216955c640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2164075eb0 0x7f2164140990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:12.697 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.693+0000 7f2169d5d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2164146e90 0x7f21641472c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:60522/0 (socket says 172.21.15.170:60522) 2024-05-08T04:08:12.697 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.693+0000 7f216955c640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f2164075eb0 0x7f2164140990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48482/0 (socket says 172.21.15.170:48482) 2024-05-08T04:08:12.697 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.693+0000 7f2169d5d640 1 -- 172.21.15.170:0/1694309521 learned_addr learned my addr 172.21.15.170:0/1694309521 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:12.697 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.693+0000 7f2168d5b640 1 --2- 172.21.15.170:0/1694309521 >> v2:172.21.15.187:3300/0 conn(0x7f21640b15d0 0x7f2164140ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:12.697 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.694+0000 7f216955c640 1 -- 172.21.15.170:0/1694309521 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2164146e90 msgr2=0x7f21641472c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:12.697 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.694+0000 7f216955c640 1 --2- 172.21.15.170:0/1694309521 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2164146e90 0x7f21641472c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:12.697 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.694+0000 7f216955c640 1 -- 172.21.15.170:0/1694309521 >> v2:172.21.15.187:3300/0 conn(0x7f21640b15d0 msgr2=0x7f2164140ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:12.697 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.694+0000 7f216955c640 1 --2- 172.21.15.170:0/1694309521 >> v2:172.21.15.187:3300/0 conn(0x7f21640b15d0 0x7f2164140ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:12.697 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.694+0000 7f216955c640 1 -- 172.21.15.170:0/1694309521 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21640b0960 con 0x7f2164075eb0 2024-05-08T04:08:12.697 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.694+0000 7f2169d5d640 1 --2- 172.21.15.170:0/1694309521 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2164146e90 0x7f21641472c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:08:12.697 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.694+0000 7f2168d5b640 1 --2- 172.21.15.170:0/1694309521 >> v2:172.21.15.187:3300/0 conn(0x7f21640b15d0 0x7f2164140ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:08:12.697 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.694+0000 7f216955c640 1 --2- 172.21.15.170:0/1694309521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2164075eb0 0x7f2164140990 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f21640b32f0 tx=0x7f216002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d271cbdb9039a84d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:12.700 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.695+0000 7f215bfff640 1 -- 172.21.15.170:0/1694309521 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21600315a0 con 0x7f2164075eb0 2024-05-08T04:08:12.700 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.695+0000 7f215bfff640 1 -- 172.21.15.170:0/1694309521 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2160037070 con 0x7f2164075eb0 2024-05-08T04:08:12.700 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.695+0000 7f215bfff640 1 -- 172.21.15.170:0/1694309521 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2160031960 con 0x7f2164075eb0 2024-05-08T04:08:12.700 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.695+0000 7f216b7c0640 1 -- 172.21.15.170:0/1694309521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2164075eb0 msgr2=0x7f2164140990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:12.701 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.695+0000 7f216b7c0640 1 --2- 172.21.15.170:0/1694309521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2164075eb0 0x7f2164140990 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f21640b32f0 tx=0x7f216002b840 comp rx=0 tx=0).stop 2024-05-08T04:08:12.701 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.695+0000 7f216b7c0640 1 -- 172.21.15.170:0/1694309521 shutdown_connections 2024-05-08T04:08:12.701 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.695+0000 7f216b7c0640 1 --2- 172.21.15.170:0/1694309521 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2164146e90 0x7f21641472c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:12.701 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.695+0000 7f216b7c0640 1 --2- 172.21.15.170:0/1694309521 >> v2:172.21.15.187:3300/0 conn(0x7f21640b15d0 0x7f2164140ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:12.701 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.695+0000 7f216b7c0640 1 --2- 172.21.15.170:0/1694309521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2164075eb0 0x7f2164140990 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:12.701 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.695+0000 7f216b7c0640 1 -- 172.21.15.170:0/1694309521 >> 172.21.15.170:0/1694309521 conn(0x7f21640b2230 msgr2=0x7f21640b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:12.701 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.695+0000 7f216b7c0640 1 -- 172.21.15.170:0/1694309521 shutdown_connections 2024-05-08T04:08:12.701 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.695+0000 7f216b7c0640 1 -- 172.21.15.170:0/1694309521 wait complete. 2024-05-08T04:08:12.701 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.696+0000 7f216b7c0640 1 Processor -- start 2024-05-08T04:08:12.701 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.696+0000 7f216b7c0640 1 -- start start 2024-05-08T04:08:12.701 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.697+0000 7f216b7c0640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2164075eb0 0x7f2164145dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:12.701 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.697+0000 7f216b7c0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21640b15d0 0x7f21641dbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:12.701 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.697+0000 7f216b7c0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2164146e90 0x7f216413aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:12.701 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.697+0000 7f216b7c0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f21640b12a0 con 0x7f21640b15d0 2024-05-08T04:08:12.702 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.697+0000 7f216b7c0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2164147800 con 0x7f2164146e90 2024-05-08T04:08:12.702 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.697+0000 7f216b7c0640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f2164143160 con 0x7f2164075eb0 2024-05-08T04:08:12.702 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.697+0000 7f216955c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2164075eb0 0x7f2164145dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:12.702 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.697+0000 7f2168d5b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21640b15d0 0x7f21641dbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:12.702 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.697+0000 7f2168d5b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21640b15d0 0x7f21641dbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48494/0 (socket says 172.21.15.170:48494) 2024-05-08T04:08:12.702 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.697+0000 7f216955c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2164075eb0 0x7f2164145dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:60536/0 (socket says 172.21.15.170:60536) 2024-05-08T04:08:12.702 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.697+0000 7f2168d5b640 1 -- 172.21.15.170:0/433059143 learned_addr learned my addr 172.21.15.170:0/433059143 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:12.702 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.697+0000 7f2169d5d640 1 --2- 172.21.15.170:0/433059143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2164146e90 0x7f216413aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:12.702 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.697+0000 7f2168d5b640 1 -- 172.21.15.170:0/433059143 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2164075eb0 msgr2=0x7f2164145dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:12.702 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.697+0000 7f2168d5b640 1 --2- 172.21.15.170:0/433059143 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2164075eb0 0x7f2164145dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:12.702 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.697+0000 7f2168d5b640 1 -- 172.21.15.170:0/433059143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2164146e90 msgr2=0x7f216413aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:12.702 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.697+0000 7f2168d5b640 1 --2- 172.21.15.170:0/433059143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2164146e90 0x7f216413aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:12.702 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.697+0000 7f2168d5b640 1 -- 172.21.15.170:0/433059143 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21641dc7e0 con 0x7f21640b15d0 2024-05-08T04:08:12.703 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.697+0000 7f2169d5d640 1 --2- 172.21.15.170:0/433059143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2164146e90 0x7f216413aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:08:12.703 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.697+0000 7f216955c640 1 --2- 172.21.15.170:0/433059143 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2164075eb0 0x7f2164145dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:08:12.703 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.697+0000 7f2168d5b640 1 --2- 172.21.15.170:0/433059143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21640b15d0 0x7f21641dbd40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f215400e730 tx=0x7f215400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:12.703 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.697+0000 7f2159ffb640 1 -- 172.21.15.170:0/433059143 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215400f040 con 0x7f21640b15d0 2024-05-08T04:08:12.703 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.698+0000 7f2159ffb640 1 -- 172.21.15.170:0/433059143 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2154004230 con 0x7f21640b15d0 2024-05-08T04:08:12.703 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.698+0000 7f216b7c0640 1 -- 172.21.15.170:0/433059143 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21641dc280 con 0x7f21640b15d0 2024-05-08T04:08:12.703 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.698+0000 7f2159ffb640 1 -- 172.21.15.170:0/433059143 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215400ee40 con 0x7f21640b15d0 2024-05-08T04:08:12.703 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.698+0000 7f216b7c0640 1 -- 172.21.15.170:0/433059143 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f216413b210 con 0x7f21640b15d0 2024-05-08T04:08:12.703 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.698+0000 7f2159ffb640 1 -- 172.21.15.170:0/433059143 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f215400ca60 con 0x7f21640b15d0 2024-05-08T04:08:12.703 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.698+0000 7f2159ffb640 1 -- 172.21.15.170:0/433059143 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f2154003ce0 con 0x7f21640b15d0 2024-05-08T04:08:12.703 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.699+0000 7f216b7c0640 1 -- 172.21.15.170:0/433059143 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2164050740 con 0x7f21640b15d0 2024-05-08T04:08:12.704 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.702+0000 7f2159ffb640 1 -- 172.21.15.170:0/433059143 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f2154017020 con 0x7f21640b15d0 2024-05-08T04:08:12.715 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-05-08T04:08:12.713+0000 7f7ca884b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T04:08:12.750 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-08T04:08:12.748+0000 7f80f120d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T04:08:12.766 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.764+0000 7f216b7c0640 1 -- 172.21.15.170:0/433059143 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f21640008d0 con 0x7f21640b15d0 2024-05-08T04:08:12.767 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.765+0000 7f2159ffb640 1 -- 172.21.15.170:0/433059143 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f215400cd50 con 0x7f21640b15d0 2024-05-08T04:08:12.767 INFO:teuthology.misc.health.smithi170.stdout: 2024-05-08T04:08:12.767 INFO:teuthology.misc.health.smithi170.stdout:{"epoch":10,"fsid":"e5497bb3-afa8-4f24-8ef4-42911ffcc962","created":"2024-05-08T04:08:09.270498+0000","modified":"2024-05-08T04:08:11.562526+0000","last_up_change":"0.000000","last_in_change":"2024-05-08T04:08:11.562526+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6bca968b-d825-417a-b907-5312aad8dbcd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"66600f89-4532-43b1-85d3-6d8c736cfaf2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a11608b2-0871-497c-aa38-264baea6bf39","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"8caae6f8-790c-426c-af8a-3a9ef54eae48","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0e804208-3607-4d71-b771-f79fb27e7968","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f88f88db-216e-4acc-ba5a-89d91b98aeb4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"6b6840fc-1e34-4cc8-9da6-8a57e69f319b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5dbcf804-181b-4d5b-9a40-f99ddaee2701","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:08:12.768 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.766+0000 7f216b7c0640 1 -- 172.21.15.170:0/433059143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21640b15d0 msgr2=0x7f21641dbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:12.768 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.766+0000 7f216b7c0640 1 --2- 172.21.15.170:0/433059143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21640b15d0 0x7f21641dbd40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f215400e730 tx=0x7f215400ebf0 comp rx=0 tx=0).stop 2024-05-08T04:08:12.769 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.766+0000 7f216b7c0640 1 -- 172.21.15.170:0/433059143 shutdown_connections 2024-05-08T04:08:12.769 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.766+0000 7f216b7c0640 1 --2- 172.21.15.170:0/433059143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2164146e90 0x7f216413aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:12.769 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.766+0000 7f216b7c0640 1 --2- 172.21.15.170:0/433059143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21640b15d0 0x7f21641dbd40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:12.769 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.766+0000 7f216b7c0640 1 --2- 172.21.15.170:0/433059143 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f2164075eb0 0x7f2164145dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:12.769 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.766+0000 7f216b7c0640 1 -- 172.21.15.170:0/433059143 >> 172.21.15.170:0/433059143 conn(0x7f21640b2230 msgr2=0x7f21640b3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:12.769 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.766+0000 7f216b7c0640 1 -- 172.21.15.170:0/433059143 shutdown_connections 2024-05-08T04:08:12.769 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:12.766+0000 7f216b7c0640 1 -- 172.21.15.170:0/433059143 wait complete. 2024-05-08T04:08:12.778 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-08T04:08:12.876 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-05-08T04:08:12.874+0000 7f7ca884b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T04:08:12.886 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-08T04:08:12.884+0000 7f80f120d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T04:08:12.936 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-08T04:08:12.934+0000 7f80f120d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T04:08:13.006 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-08T04:08:13.004+0000 7f80f120d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T04:08:13.012 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-05-08T04:08:13.010+0000 7f7ca884b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T04:08:13.053 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-08T04:08:13.051+0000 7f80f120d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T04:08:13.063 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-05-08T04:08:13.061+0000 7f7ca884b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T04:08:13.133 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-05-08T04:08:13.131+0000 7f7ca884b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T04:08:13.178 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-05-08T04:08:13.176+0000 7f7ca884b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T04:08:13.531 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-08T04:08:13.529+0000 7f80f120d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T04:08:13.648 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-08T04:08:13.646+0000 7f80f120d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T04:08:13.659 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-05-08T04:08:13.657+0000 7f7ca884b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T04:08:13.777 INFO:tasks.ceph.mgr.y.smithi170.stderr:2024-05-08T04:08:13.775+0000 7f7ca884b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T04:08:14.661 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-08T04:08:14.659+0000 7f8090af4640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-08T04:08:14.661 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-05-08T04:08:14.660+0000 7f8090af4640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-08T04:08:15.648 INFO:tasks.ceph.osd.2.smithi170.stderr:2024-05-08T04:08:15.646+0000 7f60a9013640 -1 osd.2 0 log_to_monitors true 2024-05-08T04:08:15.654 INFO:tasks.ceph.osd.5.smithi187.stderr:2024-05-08T04:08:15.653+0000 7f6c785fc640 -1 osd.5 0 log_to_monitors true 2024-05-08T04:08:15.695 INFO:tasks.ceph.osd.7.smithi187.stderr:2024-05-08T04:08:15.694+0000 7fc94bc1b640 -1 osd.7 0 log_to_monitors true 2024-05-08T04:08:15.709 INFO:tasks.ceph.osd.3.smithi170.stderr:2024-05-08T04:08:15.707+0000 7fd33f91a640 -1 osd.3 0 log_to_monitors true 2024-05-08T04:08:15.710 INFO:tasks.ceph.osd.1.smithi170.stderr:2024-05-08T04:08:15.708+0000 7f7975ab1640 -1 osd.1 0 log_to_monitors true 2024-05-08T04:08:15.719 INFO:tasks.ceph.osd.0.smithi170.stderr:2024-05-08T04:08:15.717+0000 7f8d9120f640 -1 osd.0 0 log_to_monitors true 2024-05-08T04:08:15.724 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-05-08T04:08:15.723+0000 7fcec7e15640 -1 osd.6 0 log_to_monitors true 2024-05-08T04:08:15.733 INFO:tasks.ceph.osd.4.smithi187.stderr:2024-05-08T04:08:15.732+0000 7fba89d6e640 -1 osd.4 0 log_to_monitors true 2024-05-08T04:08:16.672 INFO:tasks.ceph.osd.2.smithi170.stderr:2024-05-08T04:08:16.670+0000 7f60a49ff640 -1 osd.2 0 waiting for initial osdmap 2024-05-08T04:08:16.677 INFO:tasks.ceph.osd.2.smithi170.stderr:2024-05-08T04:08:16.675+0000 7f60a0827640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T04:08:17.670 INFO:tasks.ceph.osd.5.smithi187.stderr:2024-05-08T04:08:17.668+0000 7f6c74592640 -1 osd.5 0 waiting for initial osdmap 2024-05-08T04:08:17.671 INFO:tasks.ceph.osd.0.smithi170.stderr:2024-05-08T04:08:17.669+0000 7f8d8cb35640 -1 osd.0 0 waiting for initial osdmap 2024-05-08T04:08:17.672 INFO:tasks.ceph.osd.1.smithi170.stderr:2024-05-08T04:08:17.671+0000 7f7971a47640 -1 osd.1 0 waiting for initial osdmap 2024-05-08T04:08:17.677 INFO:tasks.ceph.osd.0.smithi170.stderr:2024-05-08T04:08:17.675+0000 7f8d8895d640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T04:08:17.678 INFO:tasks.ceph.osd.1.smithi170.stderr:2024-05-08T04:08:17.676+0000 7f796d86f640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T04:08:17.680 INFO:tasks.ceph.osd.5.smithi187.stderr:2024-05-08T04:08:17.676+0000 7f6c703ba640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T04:08:18.266 INFO:tasks.ceph.osd.3.smithi170.stderr:2024-05-08T04:08:18.264+0000 7fd33b8b0640 -1 osd.3 0 waiting for initial osdmap 2024-05-08T04:08:18.268 INFO:tasks.ceph.osd.3.smithi170.stderr:2024-05-08T04:08:18.267+0000 7fd3336d0640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T04:08:18.509 INFO:tasks.ceph.osd.4.smithi187.stderr:2024-05-08T04:08:18.507+0000 7fba85d04640 -1 osd.4 0 waiting for initial osdmap 2024-05-08T04:08:18.513 INFO:tasks.ceph.osd.4.smithi187.stderr:2024-05-08T04:08:18.511+0000 7fba7db24640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T04:08:18.559 INFO:tasks.ceph.osd.7.smithi187.stderr:2024-05-08T04:08:18.557+0000 7fc947dfb640 -1 osd.7 0 waiting for initial osdmap 2024-05-08T04:08:18.563 INFO:tasks.ceph.osd.7.smithi187.stderr:2024-05-08T04:08:18.561+0000 7fc93f407640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T04:08:18.566 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-05-08T04:08:18.564+0000 7fcec36e8640 -1 osd.6 0 waiting for initial osdmap 2024-05-08T04:08:18.569 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-05-08T04:08:18.567+0000 7fcebb508640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T04:08:19.585 DEBUG:teuthology.orchestra.run.smithi170:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T04:08:19.690 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.688+0000 7fc7c719d640 1 Processor -- start 2024-05-08T04:08:19.691 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.688+0000 7fc7c719d640 1 -- start start 2024-05-08T04:08:19.692 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.689+0000 7fc7c719d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc7c006e050 0x7fc7c00b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:19.693 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.689+0000 7fc7c719d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc7c01476f0 0x7fc7c014e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:19.693 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.689+0000 7fc7c719d640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7c00749f0 con 0x7fc7c0074590 2024-05-08T04:08:19.693 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.689+0000 7fc7c719d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc7c006e430 con 0x7fc7c01476f0 2024-05-08T04:08:19.693 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.689+0000 7fc7c719d640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fc7c0072060 con 0x7fc7c006e050 2024-05-08T04:08:19.693 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.689+0000 7fc7c599a640 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fc7c0074590 0x7fc7c0146fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:55874/0 (socket says 172.21.15.170:55874) 2024-05-08T04:08:19.693 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.689+0000 7fc7c619b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc7c006e050 0x7fc7c00b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:19.693 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.689+0000 7fc7c599a640 1 -- 172.21.15.170:0/2226546921 learned_addr learned my addr 172.21.15.170:0/2226546921 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-05-08T04:08:19.693 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.690+0000 7fc7c699c640 1 --2- 172.21.15.170:0/2226546921 >> v2:172.21.15.187:3300/0 conn(0x7fc7c01476f0 0x7fc7c014e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:19.693 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.690+0000 7fc7c619b640 1 -- 172.21.15.170:0/2226546921 >> v2:172.21.15.187:3300/0 conn(0x7fc7c01476f0 msgr2=0x7fc7c014e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:19.693 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.690+0000 7fc7c5199640 1 -- 172.21.15.170:0/2226546921 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1989077236 0 0) 0x7fc7bc001490 con 0x7fc7c0074590 2024-05-08T04:08:19.693 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.690+0000 7fc7c619b640 1 --2- 172.21.15.170:0/2226546921 >> v2:172.21.15.187:3300/0 conn(0x7fc7c01476f0 0x7fc7c014e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:19.693 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.690+0000 7fc7c619b640 1 -- 172.21.15.170:0/2226546921 >> v1:172.21.15.170:6789/0 conn(0x7fc7c0074590 legacy=0x7fc7c0146fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:19.693 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.690+0000 7fc7c619b640 1 -- 172.21.15.170:0/2226546921 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c0072340 con 0x7fc7c006e050 2024-05-08T04:08:19.693 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.690+0000 7fc7c699c640 1 --2- 172.21.15.170:0/2226546921 >> v2:172.21.15.187:3300/0 conn(0x7fc7c01476f0 0x7fc7c014e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:08:19.694 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.690+0000 7fc7c619b640 1 --2- 172.21.15.170:0/2226546921 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc7c006e050 0x7fc7c00b19d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00097a0 tx=0x7fc7b002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70c54a720e2196df server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:19.694 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.691+0000 7fc7c5199640 1 -- 172.21.15.170:0/2226546921 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0035020 con 0x7fc7c006e050 2024-05-08T04:08:19.694 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.691+0000 7fc7c5199640 1 -- 172.21.15.170:0/2226546921 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7b0031830 con 0x7fc7c006e050 2024-05-08T04:08:19.694 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.691+0000 7fc7c5199640 1 -- 172.21.15.170:0/2226546921 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0031c00 con 0x7fc7c006e050 2024-05-08T04:08:19.697 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.691+0000 7fc7c719d640 1 -- 172.21.15.170:0/2226546921 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc7c006e050 msgr2=0x7fc7c00b19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:19.697 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.691+0000 7fc7c719d640 1 --2- 172.21.15.170:0/2226546921 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc7c006e050 0x7fc7c00b19d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00097a0 tx=0x7fc7b002b8d0 comp rx=0 tx=0).stop 2024-05-08T04:08:19.697 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.691+0000 7fc7c719d640 1 -- 172.21.15.170:0/2226546921 shutdown_connections 2024-05-08T04:08:19.697 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.691+0000 7fc7c719d640 1 --2- 172.21.15.170:0/2226546921 >> v2:172.21.15.187:3300/0 conn(0x7fc7c01476f0 0x7fc7c014e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:19.697 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.691+0000 7fc7c719d640 1 --2- 172.21.15.170:0/2226546921 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc7c006e050 0x7fc7c00b19d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:19.697 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.691+0000 7fc7c719d640 1 -- 172.21.15.170:0/2226546921 >> 172.21.15.170:0/2226546921 conn(0x7fc7c00b7180 msgr2=0x7fc7c00b7590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:19.697 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.691+0000 7fc7c719d640 1 -- 172.21.15.170:0/2226546921 shutdown_connections 2024-05-08T04:08:19.697 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.691+0000 7fc7c719d640 1 -- 172.21.15.170:0/2226546921 wait complete. 2024-05-08T04:08:19.697 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.692+0000 7fc7c719d640 1 Processor -- start 2024-05-08T04:08:19.697 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.692+0000 7fc7c719d640 1 -- start start 2024-05-08T04:08:19.697 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.693+0000 7fc7c719d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc7c006e050 0x7fc7c00aed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:19.697 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.693+0000 7fc7c719d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7c0074590 0x7fc7c01dc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:19.697 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.693+0000 7fc7c719d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7c01476f0 0x7fc7c00acac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:19.697 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.693+0000 7fc7c719d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c006e6e0 con 0x7fc7c0074590 2024-05-08T04:08:19.697 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.693+0000 7fc7c719d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c0073d30 con 0x7fc7c01476f0 2024-05-08T04:08:19.698 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.693+0000 7fc7c719d640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fc7c0073eb0 con 0x7fc7c006e050 2024-05-08T04:08:19.698 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.693+0000 7fc7c619b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc7c006e050 0x7fc7c00aed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:19.698 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.693+0000 7fc7c599a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7c0074590 0x7fc7c01dc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:19.698 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.693+0000 7fc7c619b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc7c006e050 0x7fc7c00aed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:54146/0 (socket says 172.21.15.170:54146) 2024-05-08T04:08:19.698 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.693+0000 7fc7c599a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7c0074590 0x7fc7c01dc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48920/0 (socket says 172.21.15.170:48920) 2024-05-08T04:08:19.698 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.693+0000 7fc7c619b640 1 -- 172.21.15.170:0/2771294071 learned_addr learned my addr 172.21.15.170:0/2771294071 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:19.698 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.693+0000 7fc7c699c640 1 --2- 172.21.15.170:0/2771294071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7c01476f0 0x7fc7c00acac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:19.698 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.693+0000 7fc7c619b640 1 -- 172.21.15.170:0/2771294071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7c01476f0 msgr2=0x7fc7c00acac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:19.698 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.693+0000 7fc7c619b640 1 --2- 172.21.15.170:0/2771294071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7c01476f0 0x7fc7c00acac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:19.698 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.693+0000 7fc7c619b640 1 -- 172.21.15.170:0/2771294071 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7c0074590 msgr2=0x7fc7c01dc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:19.698 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.693+0000 7fc7c619b640 1 --2- 172.21.15.170:0/2771294071 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7c0074590 0x7fc7c01dc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:19.698 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.693+0000 7fc7c619b640 1 -- 172.21.15.170:0/2771294071 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c00af330 con 0x7fc7c006e050 2024-05-08T04:08:19.698 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.693+0000 7fc7c699c640 1 --2- 172.21.15.170:0/2771294071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7c01476f0 0x7fc7c00acac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:08:19.698 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.693+0000 7fc7c599a640 1 --2- 172.21.15.170:0/2771294071 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7c0074590 0x7fc7c01dc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:08:19.699 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.694+0000 7fc7c619b640 1 --2- 172.21.15.170:0/2771294071 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc7c006e050 0x7fc7c00aed50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc7b002bee0 tx=0x7fc7b00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:19.699 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.694+0000 7fc7b6ffd640 1 -- 172.21.15.170:0/2771294071 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0041070 con 0x7fc7c006e050 2024-05-08T04:08:19.699 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.694+0000 7fc7b6ffd640 1 -- 172.21.15.170:0/2771294071 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7b003a950 con 0x7fc7c006e050 2024-05-08T04:08:19.699 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.694+0000 7fc7c719d640 1 -- 172.21.15.170:0/2771294071 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7c01429e0 con 0x7fc7c006e050 2024-05-08T04:08:19.699 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.694+0000 7fc7b6ffd640 1 -- 172.21.15.170:0/2771294071 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0035040 con 0x7fc7c006e050 2024-05-08T04:08:19.699 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.694+0000 7fc7c719d640 1 -- 172.21.15.170:0/2771294071 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7c01dd240 con 0x7fc7c006e050 2024-05-08T04:08:19.699 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.695+0000 7fc7c719d640 1 -- 172.21.15.170:0/2771294071 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7c00774c0 con 0x7fc7c006e050 2024-05-08T04:08:19.700 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.695+0000 7fc7b6ffd640 1 -- 172.21.15.170:0/2771294071 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 5) ==== 98110+0+0 (secure 0 0 0) 0x7fc7b004c060 con 0x7fc7c006e050 2024-05-08T04:08:19.700 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.696+0000 7fc7b6ffd640 1 --2- 172.21.15.170:0/2771294071 >> v2:172.21.15.187:6816/101007897 conn(0x7fc7a4076200 0x7fc7a40786c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:19.700 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.696+0000 7fc7b6ffd640 1 -- 172.21.15.170:0/2771294071 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(15..15 src has 1..15) ==== 3726+0+0 (secure 0 0 0) 0x7fc7b00b85f0 con 0x7fc7c006e050 2024-05-08T04:08:19.700 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.696+0000 7fc7c599a640 1 --2- 172.21.15.170:0/2771294071 >> v2:172.21.15.187:6816/101007897 conn(0x7fc7a4076200 0x7fc7a40786c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:19.700 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.696+0000 7fc7c599a640 1 --2- 172.21.15.170:0/2771294071 >> v2:172.21.15.187:6816/101007897 conn(0x7fc7a4076200 0x7fc7a40786c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc7c01d9ea0 tx=0x7fc7bc007e90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:19.702 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.700+0000 7fc7b6ffd640 1 -- 172.21.15.170:0/2771294071 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc7b0082880 con 0x7fc7c006e050 2024-05-08T04:08:19.891 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.889+0000 7fc7c719d640 1 -- 172.21.15.170:0/2771294071 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc7c00b4870 con 0x7fc7c006e050 2024-05-08T04:08:19.892 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.890+0000 7fc7b6ffd640 1 -- 172.21.15.170:0/2771294071 <== mon.2 v2:172.21.15.170:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) ==== 74+0+10988 (secure 0 0 0) 0x7fc7b0082180 con 0x7fc7c006e050 2024-05-08T04:08:19.892 INFO:teuthology.misc.health.smithi170.stdout: 2024-05-08T04:08:19.892 INFO:teuthology.misc.health.smithi170.stdout:{"epoch":15,"fsid":"e5497bb3-afa8-4f24-8ef4-42911ffcc962","created":"2024-05-08T04:08:09.270498+0000","modified":"2024-05-08T04:08:19.679313+0000","last_up_change":"2024-05-08T04:08:18.673075+0000","last_in_change":"2024-05-08T04:08:11.562526+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:08:19.670334+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6bca968b-d825-417a-b907-5312aad8dbcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1037021465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6801","nonce":1037021465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6803","nonce":1037021465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1037021465}]},"public_addr":"172.21.15.170:6800/1037021465","cluster_addr":"172.21.15.170:6801/1037021465","heartbeat_back_addr":"172.21.15.170:6803/1037021465","heartbeat_front_addr":"172.21.15.170:6802/1037021465","state":["exists","up"]},{"osd":1,"uuid":"66600f89-4532-43b1-85d3-6d8c736cfaf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3364584304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6809","nonce":3364584304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6811","nonce":3364584304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3364584304}]},"public_addr":"172.21.15.170:6808/3364584304","cluster_addr":"172.21.15.170:6809/3364584304","heartbeat_back_addr":"172.21.15.170:6811/3364584304","heartbeat_front_addr":"172.21.15.170:6810/3364584304","state":["exists","up"]},{"osd":2,"uuid":"a11608b2-0871-497c-aa38-264baea6bf39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":3333432230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6805","nonce":3333432230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6807","nonce":3333432230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":3333432230}]},"public_addr":"172.21.15.170:6804/3333432230","cluster_addr":"172.21.15.170:6805/3333432230","heartbeat_back_addr":"172.21.15.170:6807/3333432230","heartbeat_front_addr":"172.21.15.170:6806/3333432230","state":["exists","up"]},{"osd":3,"uuid":"8caae6f8-790c-426c-af8a-3a9ef54eae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":153555120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6813","nonce":153555120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6815","nonce":153555120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":153555120}]},"public_addr":"172.21.15.170:6812/153555120","cluster_addr":"172.21.15.170:6813/153555120","heartbeat_back_addr":"172.21.15.170:6815/153555120","heartbeat_front_addr":"172.21.15.170:6814/153555120","state":["exists","up"]},{"osd":4,"uuid":"0e804208-3607-4d71-b771-f79fb27e7968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1425849822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1425849822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1425849822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":1425849822}]},"public_addr":"172.21.15.187:6808/1425849822","cluster_addr":"172.21.15.187:6811/1425849822","heartbeat_back_addr":"172.21.15.187:6815/1425849822","heartbeat_front_addr":"172.21.15.187:6813/1425849822","state":["exists","up"]},{"osd":5,"uuid":"f88f88db-216e-4acc-ba5a-89d91b98aeb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":3273856499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":3273856499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6803","nonce":3273856499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":3273856499}]},"public_addr":"172.21.15.187:6800/3273856499","cluster_addr":"172.21.15.187:6801/3273856499","heartbeat_back_addr":"172.21.15.187:6803/3273856499","heartbeat_front_addr":"172.21.15.187:6802/3273856499","state":["exists","up"]},{"osd":6,"uuid":"6b6840fc-1e34-4cc8-9da6-8a57e69f319b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":2911331731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2911331731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":2911331731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":2911331731}]},"public_addr":"172.21.15.187:6809/2911331731","cluster_addr":"172.21.15.187:6810/2911331731","heartbeat_back_addr":"172.21.15.187:6814/2911331731","heartbeat_front_addr":"172.21.15.187:6812/2911331731","state":["exists","up"]},{"osd":7,"uuid":"5dbcf804-181b-4d5b-9a40-f99ddaee2701","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2815417961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":2815417961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6807","nonce":2815417961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2815417961}]},"public_addr":"172.21.15.187:6804/2815417961","cluster_addr":"172.21.15.187:6805/2815417961","heartbeat_back_addr":"172.21.15.187:6807/2815417961","heartbeat_front_addr":"172.21.15.187:6806/2815417961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:08:16.738630+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:08:16.681089+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:08:16.630258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:08:16.670263+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:08:16.737547+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:08:16.673870+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:08:16.706810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:08:16.658743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:08:19.894 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.892+0000 7fc7c719d640 1 -- 172.21.15.170:0/2771294071 >> v2:172.21.15.187:6816/101007897 conn(0x7fc7a4076200 msgr2=0x7fc7a40786c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:19.895 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.892+0000 7fc7c719d640 1 --2- 172.21.15.170:0/2771294071 >> v2:172.21.15.187:6816/101007897 conn(0x7fc7a4076200 0x7fc7a40786c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc7c01d9ea0 tx=0x7fc7bc007e90 comp rx=0 tx=0).stop 2024-05-08T04:08:19.895 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.892+0000 7fc7c719d640 1 -- 172.21.15.170:0/2771294071 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc7c006e050 msgr2=0x7fc7c00aed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:19.895 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.892+0000 7fc7c719d640 1 --2- 172.21.15.170:0/2771294071 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc7c006e050 0x7fc7c00aed50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc7b002bee0 tx=0x7fc7b00046c0 comp rx=0 tx=0).stop 2024-05-08T04:08:19.895 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.892+0000 7fc7c719d640 1 -- 172.21.15.170:0/2771294071 shutdown_connections 2024-05-08T04:08:19.895 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.892+0000 7fc7c719d640 1 --2- 172.21.15.170:0/2771294071 >> v2:172.21.15.187:6816/101007897 conn(0x7fc7a4076200 0x7fc7a40786c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:19.895 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.892+0000 7fc7c719d640 1 --2- 172.21.15.170:0/2771294071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7c01476f0 0x7fc7c00acac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:19.895 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.892+0000 7fc7c719d640 1 --2- 172.21.15.170:0/2771294071 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7c0074590 0x7fc7c01dc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:19.895 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.892+0000 7fc7c719d640 1 --2- 172.21.15.170:0/2771294071 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc7c006e050 0x7fc7c00aed50 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:19.895 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.892+0000 7fc7c719d640 1 -- 172.21.15.170:0/2771294071 >> 172.21.15.170:0/2771294071 conn(0x7fc7c00b7180 msgr2=0x7fc7c013f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:19.895 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.892+0000 7fc7c719d640 1 -- 172.21.15.170:0/2771294071 shutdown_connections 2024-05-08T04:08:19.895 INFO:teuthology.misc.health.smithi170.stderr:2024-05-08T04:08:19.892+0000 7fc7c719d640 1 -- 172.21.15.170:0/2771294071 wait complete. 2024-05-08T04:08:19.904 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-08T04:08:19.904 INFO:tasks.ceph:Creating RBD pool 2024-05-08T04:08:19.904 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd pool create rbd 8 2024-05-08T04:08:20.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.000+0000 7f03be49c640 1 Processor -- start 2024-05-08T04:08:20.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.000+0000 7f03be49c640 1 -- start start 2024-05-08T04:08:20.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.000+0000 7f03be49c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f03b80726a0 0x7f03b8072a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:20.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.000+0000 7f03be49c640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f03b80b1df0 0x7f03b8141c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:20.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.000+0000 7f03be49c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f03b81421c0 0x7f03b813e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:20.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.000+0000 7f03be49c640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7f03b81417f0 con 0x7f03b80b1df0 2024-05-08T04:08:20.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.000+0000 7f03be49c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f03b8073a80 con 0x7f03b80726a0 2024-05-08T04:08:20.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.000+0000 7f03be49c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f03b8074610 con 0x7f03b81421c0 2024-05-08T04:08:20.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.001+0000 7f03b77fe640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f03b80b1df0 0x7f03b8141c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:20.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.001+0000 7f03bca39640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f03b81421c0 0x7f03b813e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:20.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.001+0000 7f03b77fe640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f03b80b1df0 0x7f03b8141c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48924/0 (socket says 172.21.15.170:48924) 2024-05-08T04:08:20.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.001+0000 7f03bca39640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f03b81421c0 0x7f03b813e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:54156/0 (socket says 172.21.15.170:54156) 2024-05-08T04:08:20.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.001+0000 7f03b77fe640 1 -- 172.21.15.170:0/4124446678 learned_addr learned my addr 172.21.15.170:0/4124446678 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:20.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.001+0000 7f03b7fff640 1 --2- 172.21.15.170:0/4124446678 >> v2:172.21.15.187:3300/0 conn(0x7f03b80726a0 0x7f03b8072a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:20.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.001+0000 7f03b77fe640 1 -- 172.21.15.170:0/4124446678 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f03b81421c0 msgr2=0x7f03b813e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:20.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.001+0000 7f03b77fe640 1 --2- 172.21.15.170:0/4124446678 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f03b81421c0 0x7f03b813e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:20.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.001+0000 7f03b77fe640 1 -- 172.21.15.170:0/4124446678 >> v2:172.21.15.187:3300/0 conn(0x7f03b80726a0 msgr2=0x7f03b8072a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:20.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.001+0000 7f03b77fe640 1 --2- 172.21.15.170:0/4124446678 >> v2:172.21.15.187:3300/0 conn(0x7f03b80726a0 0x7f03b8072a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:20.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.001+0000 7f03b77fe640 1 -- 172.21.15.170:0/4124446678 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b80aa650 con 0x7f03b80b1df0 2024-05-08T04:08:20.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.001+0000 7f03bca39640 1 --2- 172.21.15.170:0/4124446678 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f03b81421c0 0x7f03b813e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:08:20.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.002+0000 7f03b77fe640 1 --2- 172.21.15.170:0/4124446678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03b80b1df0 0x7f03b8141c80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f03b80b49f0 tx=0x7f03ac02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6365ead9e2a4dd92 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:20.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.002+0000 7f03b6ffd640 1 -- 172.21.15.170:0/4124446678 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac035020 con 0x7f03b80b1df0 2024-05-08T04:08:20.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.002+0000 7f03b6ffd640 1 -- 172.21.15.170:0/4124446678 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03ac031870 con 0x7f03b80b1df0 2024-05-08T04:08:20.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.002+0000 7f03b6ffd640 1 -- 172.21.15.170:0/4124446678 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac031c40 con 0x7f03b80b1df0 2024-05-08T04:08:20.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.002+0000 7f03be49c640 1 -- 172.21.15.170:0/4124446678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03b80b1df0 msgr2=0x7f03b8141c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:20.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.002+0000 7f03be49c640 1 --2- 172.21.15.170:0/4124446678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03b80b1df0 0x7f03b8141c80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f03b80b49f0 tx=0x7f03ac02b910 comp rx=0 tx=0).stop 2024-05-08T04:08:20.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.002+0000 7f03be49c640 1 -- 172.21.15.170:0/4124446678 shutdown_connections 2024-05-08T04:08:20.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.002+0000 7f03be49c640 1 --2- 172.21.15.170:0/4124446678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03b80b1df0 0x7f03b8141c80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:20.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.002+0000 7f03be49c640 1 --2- 172.21.15.170:0/4124446678 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f03b81421c0 0x7f03b813e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:20.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.002+0000 7f03be49c640 1 --2- 172.21.15.170:0/4124446678 >> v2:172.21.15.187:3300/0 conn(0x7f03b80726a0 0x7f03b8072a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:20.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.002+0000 7f03be49c640 1 -- 172.21.15.170:0/4124446678 >> 172.21.15.170:0/4124446678 conn(0x7f03b80b3de0 msgr2=0x7f03b80b41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:20.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.002+0000 7f03be49c640 1 -- 172.21.15.170:0/4124446678 shutdown_connections 2024-05-08T04:08:20.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.002+0000 7f03be49c640 1 -- 172.21.15.170:0/4124446678 wait complete. 2024-05-08T04:08:20.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.003+0000 7f03be49c640 1 Processor -- start 2024-05-08T04:08:20.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.004+0000 7f03be49c640 1 -- start start 2024-05-08T04:08:20.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.004+0000 7f03be49c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03b80726a0 0x7f03b813d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:20.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.004+0000 7f03be49c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f03b80b1df0 0x7f03b81370d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:20.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.004+0000 7f03be49c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03b81421c0 0x7f03b8137610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:20.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.004+0000 7f03be49c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f03b813f0a0 con 0x7f03b81421c0 2024-05-08T04:08:20.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.004+0000 7f03be49c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f03b813ef20 con 0x7f03b80726a0 2024-05-08T04:08:20.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.004+0000 7f03be49c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f03b813f220 con 0x7f03b80b1df0 2024-05-08T04:08:20.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.004+0000 7f03b77fe640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f03b80b1df0 0x7f03b81370d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:20.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.004+0000 7f03bca39640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03b81421c0 0x7f03b8137610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:20.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.004+0000 7f03bca39640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03b81421c0 0x7f03b8137610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48932/0 (socket says 172.21.15.170:48932) 2024-05-08T04:08:20.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.004+0000 7f03bca39640 1 -- 172.21.15.170:0/1491998448 learned_addr learned my addr 172.21.15.170:0/1491998448 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:20.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.004+0000 7f03b7fff640 1 --2- 172.21.15.170:0/1491998448 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03b80726a0 0x7f03b813d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:20.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.004+0000 7f03b77fe640 1 -- 172.21.15.170:0/1491998448 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03b80726a0 msgr2=0x7f03b813d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:20.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.004+0000 7f03b77fe640 1 --2- 172.21.15.170:0/1491998448 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03b80726a0 0x7f03b813d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:20.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.004+0000 7f03b77fe640 1 -- 172.21.15.170:0/1491998448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03b81421c0 msgr2=0x7f03b8137610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:20.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.004+0000 7f03b77fe640 1 --2- 172.21.15.170:0/1491998448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03b81421c0 0x7f03b8137610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:20.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.004+0000 7f03b77fe640 1 -- 172.21.15.170:0/1491998448 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b806d720 con 0x7f03b80b1df0 2024-05-08T04:08:20.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.004+0000 7f03bca39640 1 --2- 172.21.15.170:0/1491998448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03b81421c0 0x7f03b8137610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:08:20.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.005+0000 7f03b77fe640 1 --2- 172.21.15.170:0/1491998448 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f03b80b1df0 0x7f03b81370d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f03ac033000 tx=0x7f03ac0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:20.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.005+0000 7f03b4ff9640 1 -- 172.21.15.170:0/1491998448 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac040070 con 0x7f03b80b1df0 2024-05-08T04:08:20.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.005+0000 7f03b4ff9640 1 -- 172.21.15.170:0/1491998448 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03ac030d00 con 0x7f03b80b1df0 2024-05-08T04:08:20.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.005+0000 7f03be49c640 1 -- 172.21.15.170:0/1491998448 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03b80b5330 con 0x7f03b80b1df0 2024-05-08T04:08:20.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.005+0000 7f03b4ff9640 1 -- 172.21.15.170:0/1491998448 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac035040 con 0x7f03b80b1df0 2024-05-08T04:08:20.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.005+0000 7f03be49c640 1 -- 172.21.15.170:0/1491998448 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f03b80b6ba0 con 0x7f03b80b1df0 2024-05-08T04:08:20.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.006+0000 7f03be49c640 1 -- 172.21.15.170:0/1491998448 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03b80b2ef0 con 0x7f03b80b1df0 2024-05-08T04:08:20.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.006+0000 7f03b4ff9640 1 -- 172.21.15.170:0/1491998448 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 5) ==== 98110+0+0 (secure 0 0 0) 0x7f03ac04b060 con 0x7f03b80b1df0 2024-05-08T04:08:20.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.007+0000 7f03b4ff9640 1 --2- 172.21.15.170:0/1491998448 >> v2:172.21.15.187:6816/101007897 conn(0x7f03a4076200 0x7f03a40786c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:20.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.007+0000 7f03b4ff9640 1 -- 172.21.15.170:0/1491998448 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(15..15 src has 1..15) ==== 3726+0+0 (secure 0 0 0) 0x7f03ac0b8a10 con 0x7f03b80b1df0 2024-05-08T04:08:20.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.007+0000 7f03b7fff640 1 --2- 172.21.15.170:0/1491998448 >> v2:172.21.15.187:6816/101007897 conn(0x7f03a4076200 0x7f03a40786c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:20.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.008+0000 7f03b7fff640 1 --2- 172.21.15.170:0/1491998448 >> v2:172.21.15.187:6816/101007897 conn(0x7f03a4076200 0x7f03a40786c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f03a8006fd0 tx=0x7f03a8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:20.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.011+0000 7f03b4ff9640 1 -- 172.21.15.170:0/1491998448 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f03b80b2ef0 con 0x7f03b80b1df0 2024-05-08T04:08:20.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.192+0000 7f03be49c640 1 -- 172.21.15.170:0/1491998448 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) -- 0x7f03b806b400 con 0x7f03b80b1df0 2024-05-08T04:08:20.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.686+0000 7f03b4ff9640 1 -- 172.21.15.170:0/1491998448 <== mon.2 v2:172.21.15.170:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v16) ==== 109+0+0 (secure 0 0 0) 0x7f03ac082b80 con 0x7f03b80b1df0 2024-05-08T04:08:20.688 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'rbd' created 2024-05-08T04:08:20.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.690+0000 7f03be49c640 1 -- 172.21.15.170:0/1491998448 >> v2:172.21.15.187:6816/101007897 conn(0x7f03a4076200 msgr2=0x7f03a40786c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:20.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.690+0000 7f03be49c640 1 --2- 172.21.15.170:0/1491998448 >> v2:172.21.15.187:6816/101007897 conn(0x7f03a4076200 0x7f03a40786c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f03a8006fd0 tx=0x7f03a8008040 comp rx=0 tx=0).stop 2024-05-08T04:08:20.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.690+0000 7f03be49c640 1 -- 172.21.15.170:0/1491998448 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f03b80b1df0 msgr2=0x7f03b81370d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:20.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.690+0000 7f03be49c640 1 --2- 172.21.15.170:0/1491998448 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f03b80b1df0 0x7f03b81370d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f03ac033000 tx=0x7f03ac0046c0 comp rx=0 tx=0).stop 2024-05-08T04:08:20.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.690+0000 7f03be49c640 1 -- 172.21.15.170:0/1491998448 shutdown_connections 2024-05-08T04:08:20.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.690+0000 7f03be49c640 1 --2- 172.21.15.170:0/1491998448 >> v2:172.21.15.187:6816/101007897 conn(0x7f03a4076200 0x7f03a40786c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:20.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.690+0000 7f03be49c640 1 --2- 172.21.15.170:0/1491998448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03b81421c0 0x7f03b8137610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:20.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.690+0000 7f03be49c640 1 --2- 172.21.15.170:0/1491998448 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f03b80b1df0 0x7f03b81370d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:20.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.690+0000 7f03be49c640 1 --2- 172.21.15.170:0/1491998448 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03b80726a0 0x7f03b813d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:20.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.690+0000 7f03be49c640 1 -- 172.21.15.170:0/1491998448 >> 172.21.15.170:0/1491998448 conn(0x7f03b80b3de0 msgr2=0x7f03b8142960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:20.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.691+0000 7f03be49c640 1 -- 172.21.15.170:0/1491998448 shutdown_connections 2024-05-08T04:08:20.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.691+0000 7f03be49c640 1 -- 172.21.15.170:0/1491998448 wait complete. 2024-05-08T04:08:20.704 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-05-08T04:08:20.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.804+0000 7f0ecf0bf640 1 Processor -- start 2024-05-08T04:08:20.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.805+0000 7f0ecf0bf640 1 -- start start 2024-05-08T04:08:20.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.805+0000 7f0ecf0bf640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0ec80b0750 0x7f0ec8139500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:20.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.805+0000 7f0ecf0bf640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0ec81437a0 0x7f0ec814a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:20.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.805+0000 7f0ecf0bf640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ec813d650 con 0x7f0ec8139a40 2024-05-08T04:08:20.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.805+0000 7f0ecf0bf640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7f0ec80b0b30 con 0x7f0ec80b0750 2024-05-08T04:08:20.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.805+0000 7f0ecf0bf640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f0ec8074070 con 0x7f0ec81437a0 2024-05-08T04:08:20.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.805+0000 7f0ecce5b640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0ec80b0750 0x7f0ec8139500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:20.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.805+0000 7f0ec7fff640 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f0ec8139a40 0x7f0ec8143090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:55886/0 (socket says 172.21.15.170:55886) 2024-05-08T04:08:20.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.805+0000 7f0ec7fff640 1 -- 172.21.15.170:0/2926766098 learned_addr learned my addr 172.21.15.170:0/2926766098 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-05-08T04:08:20.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.806+0000 7f0ecd65c640 1 --2- 172.21.15.170:0/2926766098 >> v2:172.21.15.187:3300/0 conn(0x7f0ec81437a0 0x7f0ec814a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:20.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.806+0000 7f0ecce5b640 1 -- 172.21.15.170:0/2926766098 >> v2:172.21.15.187:3300/0 conn(0x7f0ec81437a0 msgr2=0x7f0ec814a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:20.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.806+0000 7f0ecce5b640 1 --2- 172.21.15.170:0/2926766098 >> v2:172.21.15.187:3300/0 conn(0x7f0ec81437a0 0x7f0ec814a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:20.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.806+0000 7f0ecce5b640 1 -- 172.21.15.170:0/2926766098 >> v1:172.21.15.170:6789/0 conn(0x7f0ec8139a40 legacy=0x7f0ec8143090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:20.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.806+0000 7f0ecce5b640 1 -- 172.21.15.170:0/2926766098 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ec8073b50 con 0x7f0ec80b0750 2024-05-08T04:08:20.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.806+0000 7f0ecce5b640 1 --2- 172.21.15.170:0/2926766098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ec80b0750 0x7f0ec8139500 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8072390 tx=0x7f0ec002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd264f6687c050e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:20.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.806+0000 7f0ec77fe640 1 -- 172.21.15.170:0/2926766098 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec0035020 con 0x7f0ec80b0750 2024-05-08T04:08:20.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.807+0000 7f0ec77fe640 1 -- 172.21.15.170:0/2926766098 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ec0031830 con 0x7f0ec80b0750 2024-05-08T04:08:20.809 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.807+0000 7f0ec77fe640 1 -- 172.21.15.170:0/2926766098 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec0031c00 con 0x7f0ec80b0750 2024-05-08T04:08:20.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.807+0000 7f0ecf0bf640 1 -- 172.21.15.170:0/2926766098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ec80b0750 msgr2=0x7f0ec8139500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:20.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.807+0000 7f0ecf0bf640 1 --2- 172.21.15.170:0/2926766098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ec80b0750 0x7f0ec8139500 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8072390 tx=0x7f0ec002b8d0 comp rx=0 tx=0).stop 2024-05-08T04:08:20.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.807+0000 7f0ecf0bf640 1 -- 172.21.15.170:0/2926766098 shutdown_connections 2024-05-08T04:08:20.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.807+0000 7f0ecf0bf640 1 --2- 172.21.15.170:0/2926766098 >> v2:172.21.15.187:3300/0 conn(0x7f0ec81437a0 0x7f0ec814a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:20.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.807+0000 7f0ecf0bf640 1 --2- 172.21.15.170:0/2926766098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ec80b0750 0x7f0ec8139500 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:20.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.807+0000 7f0ecf0bf640 1 -- 172.21.15.170:0/2926766098 >> 172.21.15.170:0/2926766098 conn(0x7f0ec80b5250 msgr2=0x7f0ec80b5660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:20.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.807+0000 7f0ecf0bf640 1 -- 172.21.15.170:0/2926766098 shutdown_connections 2024-05-08T04:08:20.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.807+0000 7f0ecf0bf640 1 -- 172.21.15.170:0/2926766098 wait complete. 2024-05-08T04:08:20.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.808+0000 7f0ecf0bf640 1 Processor -- start 2024-05-08T04:08:20.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.808+0000 7f0ecf0bf640 1 -- start start 2024-05-08T04:08:20.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.809+0000 7f0ecf0bf640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ec80b0750 0x7f0ec81d7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:20.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.809+0000 7f0ecf0bf640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0ec8139a40 0x7f0ec81d4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:20.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.809+0000 7f0ecf0bf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ec81437a0 0x7f0ec81d4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:20.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.809+0000 7f0ecf0bf640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec8139e50 con 0x7f0ec81437a0 2024-05-08T04:08:20.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.809+0000 7f0ecf0bf640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec8143bb0 con 0x7f0ec80b0750 2024-05-08T04:08:20.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.809+0000 7f0ecf0bf640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f0ec8074440 con 0x7f0ec8139a40 2024-05-08T04:08:20.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.809+0000 7f0ecd65c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ec81437a0 0x7f0ec81d4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:20.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.809+0000 7f0ecd65c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ec81437a0 0x7f0ec81d4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48954/0 (socket says 172.21.15.170:48954) 2024-05-08T04:08:20.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.809+0000 7f0ecd65c640 1 -- 172.21.15.170:0/2961605528 learned_addr learned my addr 172.21.15.170:0/2961605528 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:20.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.809+0000 7f0ec7fff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0ec8139a40 0x7f0ec81d4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:20.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.809+0000 7f0ecce5b640 1 --2- 172.21.15.170:0/2961605528 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ec80b0750 0x7f0ec81d7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:20.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.809+0000 7f0ecd65c640 1 -- 172.21.15.170:0/2961605528 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0ec8139a40 msgr2=0x7f0ec81d4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:20.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.809+0000 7f0ecd65c640 1 --2- 172.21.15.170:0/2961605528 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0ec8139a40 0x7f0ec81d4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:20.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.809+0000 7f0ecd65c640 1 -- 172.21.15.170:0/2961605528 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ec80b0750 msgr2=0x7f0ec81d7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:20.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.809+0000 7f0ecd65c640 1 --2- 172.21.15.170:0/2961605528 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ec80b0750 0x7f0ec81d7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:20.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.809+0000 7f0ecd65c640 1 -- 172.21.15.170:0/2961605528 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ec81d4aa0 con 0x7f0ec81437a0 2024-05-08T04:08:20.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.809+0000 7f0ec7fff640 1 --2- 172.21.15.170:0/2961605528 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0ec8139a40 0x7f0ec81d4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:08:20.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.809+0000 7f0ecd65c640 1 --2- 172.21.15.170:0/2961605528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ec81437a0 0x7f0ec81d4560 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0eb800cc30 tx=0x7f0eb800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:20.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.810+0000 7f0ec57fa640 1 -- 172.21.15.170:0/2961605528 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb800ce40 con 0x7f0ec81437a0 2024-05-08T04:08:20.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.810+0000 7f0ec57fa640 1 -- 172.21.15.170:0/2961605528 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0eb8003990 con 0x7f0ec81437a0 2024-05-08T04:08:20.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.810+0000 7f0ecf0bf640 1 -- 172.21.15.170:0/2961605528 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ec813d070 con 0x7f0ec81437a0 2024-05-08T04:08:20.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.810+0000 7f0ec57fa640 1 -- 172.21.15.170:0/2961605528 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb8003d40 con 0x7f0ec81437a0 2024-05-08T04:08:20.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.810+0000 7f0ecf0bf640 1 -- 172.21.15.170:0/2961605528 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ec81d3b80 con 0x7f0ec81437a0 2024-05-08T04:08:20.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.811+0000 7f0ecf0bf640 1 -- 172.21.15.170:0/2961605528 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ec806b400 con 0x7f0ec81437a0 2024-05-08T04:08:20.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.811+0000 7f0ec57fa640 1 -- 172.21.15.170:0/2961605528 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 5) ==== 98110+0+0 (secure 0 0 0) 0x7f0eb8004000 con 0x7f0ec81437a0 2024-05-08T04:08:20.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.812+0000 7f0ec57fa640 1 --2- 172.21.15.170:0/2961605528 >> v2:172.21.15.187:6816/101007897 conn(0x7f0ea8076200 0x7f0ea80786c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:20.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.812+0000 7f0ec57fa640 1 -- 172.21.15.170:0/2961605528 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4079+0+0 (secure 0 0 0) 0x7f0eb8097140 con 0x7f0ec81437a0 2024-05-08T04:08:20.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.812+0000 7f0ecce5b640 1 --2- 172.21.15.170:0/2961605528 >> v2:172.21.15.187:6816/101007897 conn(0x7f0ea8076200 0x7f0ea80786c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:20.815 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.813+0000 7f0ecce5b640 1 --2- 172.21.15.170:0/2961605528 >> v2:172.21.15.187:6816/101007897 conn(0x7f0ea8076200 0x7f0ea80786c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0036000 tx=0x7f0ec0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:20.818 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:20.817+0000 7f0ec57fa640 1 -- 172.21.15.170:0/2961605528 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0eb8014030 con 0x7f0ec81437a0 2024-05-08T04:08:21.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.016+0000 7f0ecf0bf640 1 -- 172.21.15.170:0/2961605528 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) -- 0x7f0ec81d71f0 con 0x7f0ec81437a0 2024-05-08T04:08:21.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.685+0000 7f0ec57fa640 1 -- 172.21.15.170:0/2961605528 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v17) ==== 173+0+0 (secure 0 0 0) 0x7f0eb8060c20 con 0x7f0ec81437a0 2024-05-08T04:08:21.687 INFO:teuthology.orchestra.run.smithi170.stderr:enabled application 'rbd' on pool 'rbd' 2024-05-08T04:08:21.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.688+0000 7f0ecf0bf640 1 -- 172.21.15.170:0/2961605528 >> v2:172.21.15.187:6816/101007897 conn(0x7f0ea8076200 msgr2=0x7f0ea80786c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:21.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.688+0000 7f0ecf0bf640 1 --2- 172.21.15.170:0/2961605528 >> v2:172.21.15.187:6816/101007897 conn(0x7f0ea8076200 0x7f0ea80786c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0036000 tx=0x7f0ec0008030 comp rx=0 tx=0).stop 2024-05-08T04:08:21.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.688+0000 7f0ecf0bf640 1 -- 172.21.15.170:0/2961605528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ec81437a0 msgr2=0x7f0ec81d4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:21.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.688+0000 7f0ecf0bf640 1 --2- 172.21.15.170:0/2961605528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ec81437a0 0x7f0ec81d4560 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0eb800cc30 tx=0x7f0eb800a5c0 comp rx=0 tx=0).stop 2024-05-08T04:08:21.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.688+0000 7f0ecf0bf640 1 -- 172.21.15.170:0/2961605528 shutdown_connections 2024-05-08T04:08:21.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.688+0000 7f0ecf0bf640 1 --2- 172.21.15.170:0/2961605528 >> v2:172.21.15.187:6816/101007897 conn(0x7f0ea8076200 0x7f0ea80786c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:21.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.688+0000 7f0ecf0bf640 1 --2- 172.21.15.170:0/2961605528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0ec81437a0 0x7f0ec81d4560 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:21.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.688+0000 7f0ecf0bf640 1 --2- 172.21.15.170:0/2961605528 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0ec8139a40 0x7f0ec81d4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:21.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.688+0000 7f0ecf0bf640 1 --2- 172.21.15.170:0/2961605528 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0ec80b0750 0x7f0ec81d7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:21.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.688+0000 7f0ecf0bf640 1 -- 172.21.15.170:0/2961605528 >> 172.21.15.170:0/2961605528 conn(0x7f0ec80b5250 msgr2=0x7f0ec80b2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:21.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.688+0000 7f0ecf0bf640 1 -- 172.21.15.170:0/2961605528 shutdown_connections 2024-05-08T04:08:21.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.688+0000 7f0ecf0bf640 1 -- 172.21.15.170:0/2961605528 wait complete. 2024-05-08T04:08:21.705 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-08T04:08:21.705 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-08T04:08:21.705 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-08T04:08:21.705 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-08T04:08:21.706 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-08T04:08:21.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.803+0000 7f911d391640 1 Processor -- start 2024-05-08T04:08:21.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.803+0000 7f911d391640 1 -- start start 2024-05-08T04:08:21.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.804+0000 7f911d391640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9118075ea0 0x7f911813e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:21.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.804+0000 7f911d391640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f91180b0260 0x7f911813eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:21.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.804+0000 7f911d391640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f91181415e0 0x7f9118141a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:21.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.804+0000 7f911d391640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7f9118076280 con 0x7f91180b0260 2024-05-08T04:08:21.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.804+0000 7f911d391640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f91180b24b0 con 0x7f91181415e0 2024-05-08T04:08:21.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.804+0000 7f911d391640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f9118141f90 con 0x7f9118075ea0 2024-05-08T04:08:21.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.804+0000 7f91167fc640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f91180b0260 0x7f911813eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:21.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.804+0000 7f9116ffd640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9118075ea0 0x7f911813e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:21.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.804+0000 7f9116ffd640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9118075ea0 0x7f911813e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:54186/0 (socket says 172.21.15.170:54186) 2024-05-08T04:08:21.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.804+0000 7f91167fc640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f91180b0260 0x7f911813eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48962/0 (socket says 172.21.15.170:48962) 2024-05-08T04:08:21.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.804+0000 7f9116ffd640 1 -- 172.21.15.170:0/2143983885 learned_addr learned my addr 172.21.15.170:0/2143983885 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:21.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.804+0000 7f91177fe640 1 --2- 172.21.15.170:0/2143983885 >> v2:172.21.15.187:3300/0 conn(0x7f91181415e0 0x7f9118141a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:21.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.804+0000 7f91167fc640 1 -- 172.21.15.170:0/2143983885 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9118075ea0 msgr2=0x7f911813e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:21.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.804+0000 7f91167fc640 1 --2- 172.21.15.170:0/2143983885 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9118075ea0 0x7f911813e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:21.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.804+0000 7f91167fc640 1 -- 172.21.15.170:0/2143983885 >> v2:172.21.15.187:3300/0 conn(0x7f91181415e0 msgr2=0x7f9118141a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:21.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.804+0000 7f91167fc640 1 --2- 172.21.15.170:0/2143983885 >> v2:172.21.15.187:3300/0 conn(0x7f91181415e0 0x7f9118141a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:21.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.804+0000 7f91167fc640 1 -- 172.21.15.170:0/2143983885 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91180aa650 con 0x7f91180b0260 2024-05-08T04:08:21.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.804+0000 7f9116ffd640 1 --2- 172.21.15.170:0/2143983885 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9118075ea0 0x7f911813e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:08:21.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.805+0000 7f91167fc640 1 --2- 172.21.15.170:0/2143983885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91180b0260 0x7f911813eb00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f91180740c0 tx=0x7f910002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b659c7caa73369eb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:21.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.805+0000 7f9115ffb640 1 -- 172.21.15.170:0/2143983885 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91000315a0 con 0x7f91180b0260 2024-05-08T04:08:21.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.805+0000 7f9115ffb640 1 -- 172.21.15.170:0/2143983885 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9100037070 con 0x7f91180b0260 2024-05-08T04:08:21.808 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.805+0000 7f9115ffb640 1 -- 172.21.15.170:0/2143983885 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9100031960 con 0x7f91180b0260 2024-05-08T04:08:21.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.806+0000 7f911d391640 1 -- 172.21.15.170:0/2143983885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91180b0260 msgr2=0x7f911813eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:21.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.806+0000 7f911d391640 1 --2- 172.21.15.170:0/2143983885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91180b0260 0x7f911813eb00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f91180740c0 tx=0x7f910002b840 comp rx=0 tx=0).stop 2024-05-08T04:08:21.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.806+0000 7f911d391640 1 -- 172.21.15.170:0/2143983885 shutdown_connections 2024-05-08T04:08:21.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.806+0000 7f911d391640 1 --2- 172.21.15.170:0/2143983885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91180b0260 0x7f911813eb00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:21.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.806+0000 7f911d391640 1 --2- 172.21.15.170:0/2143983885 >> v2:172.21.15.187:3300/0 conn(0x7f91181415e0 0x7f9118141a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:21.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.806+0000 7f911d391640 1 --2- 172.21.15.170:0/2143983885 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f9118075ea0 0x7f911813e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:21.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.806+0000 7f911d391640 1 -- 172.21.15.170:0/2143983885 >> 172.21.15.170:0/2143983885 conn(0x7f91180b6020 msgr2=0x7f91180b6430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:21.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.806+0000 7f911d391640 1 -- 172.21.15.170:0/2143983885 shutdown_connections 2024-05-08T04:08:21.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.806+0000 7f911d391640 1 -- 172.21.15.170:0/2143983885 wait complete. 2024-05-08T04:08:21.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.807+0000 7f911d391640 1 Processor -- start 2024-05-08T04:08:21.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.807+0000 7f911d391640 1 -- start start 2024-05-08T04:08:21.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.808+0000 7f911d391640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9118075ea0 0x7f91180ab9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:21.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.808+0000 7f911d391640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91180b0260 0x7f91181d7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:21.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.808+0000 7f911d391640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f91181415e0 0x7f91180af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:21.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.808+0000 7f911d391640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f91180b1610 con 0x7f9118075ea0 2024-05-08T04:08:21.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.808+0000 7f911d391640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f91180b10c0 con 0x7f91180b0260 2024-05-08T04:08:21.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.808+0000 7f911d391640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f91180b1790 con 0x7f91181415e0 2024-05-08T04:08:21.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.808+0000 7f9116ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9118075ea0 0x7f91180ab9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:21.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.808+0000 7f91177fe640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f91181415e0 0x7f91180af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:21.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.808+0000 7f91177fe640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f91181415e0 0x7f91180af1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:54200/0 (socket says 172.21.15.170:54200) 2024-05-08T04:08:21.812 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.808+0000 7f91177fe640 1 -- 172.21.15.170:0/3673390289 learned_addr learned my addr 172.21.15.170:0/3673390289 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:21.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.808+0000 7f91167fc640 1 --2- 172.21.15.170:0/3673390289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91180b0260 0x7f91181d7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:21.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.808+0000 7f91177fe640 1 -- 172.21.15.170:0/3673390289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91180b0260 msgr2=0x7f91181d7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:21.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.808+0000 7f91177fe640 1 --2- 172.21.15.170:0/3673390289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91180b0260 0x7f91181d7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:21.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.808+0000 7f91177fe640 1 -- 172.21.15.170:0/3673390289 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9118075ea0 msgr2=0x7f91180ab9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:21.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.808+0000 7f91177fe640 1 --2- 172.21.15.170:0/3673390289 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9118075ea0 0x7f91180ab9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:21.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.808+0000 7f91177fe640 1 -- 172.21.15.170:0/3673390289 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91180ad7b0 con 0x7f91181415e0 2024-05-08T04:08:21.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.808+0000 7f9116ffd640 1 --2- 172.21.15.170:0/3673390289 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9118075ea0 0x7f91180ab9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:08:21.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.808+0000 7f91177fe640 1 --2- 172.21.15.170:0/3673390289 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f91181415e0 0x7f91180af1f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f910800e7f0 tx=0x7f910800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:21.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.808+0000 7f910f7fe640 1 -- 172.21.15.170:0/3673390289 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9108017070 con 0x7f91181415e0 2024-05-08T04:08:21.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.808+0000 7f910f7fe640 1 -- 172.21.15.170:0/3673390289 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f910800c730 con 0x7f91181415e0 2024-05-08T04:08:21.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.808+0000 7f910f7fe640 1 -- 172.21.15.170:0/3673390289 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910800cb20 con 0x7f91181415e0 2024-05-08T04:08:21.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.808+0000 7f911d391640 1 -- 172.21.15.170:0/3673390289 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91180ada90 con 0x7f91181415e0 2024-05-08T04:08:21.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.808+0000 7f911d391640 1 -- 172.21.15.170:0/3673390289 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f91180ab430 con 0x7f91181415e0 2024-05-08T04:08:21.813 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.809+0000 7f911d391640 1 -- 172.21.15.170:0/3673390289 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91180733e0 con 0x7f91181415e0 2024-05-08T04:08:21.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.810+0000 7f910f7fe640 1 -- 172.21.15.170:0/3673390289 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 5) ==== 98110+0+0 (secure 0 0 0) 0x7f910801a070 con 0x7f91181415e0 2024-05-08T04:08:21.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.810+0000 7f910f7fe640 1 --2- 172.21.15.170:0/3673390289 >> v2:172.21.15.187:6816/101007897 conn(0x7f90f8076200 0x7f90f80786c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:21.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.811+0000 7f910f7fe640 1 -- 172.21.15.170:0/3673390289 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(17..17 src has 1..17) ==== 4090+0+0 (secure 0 0 0) 0x7f9108097270 con 0x7f91181415e0 2024-05-08T04:08:21.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.811+0000 7f9116ffd640 1 --2- 172.21.15.170:0/3673390289 >> v2:172.21.15.187:6816/101007897 conn(0x7f90f8076200 0x7f90f80786c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:21.814 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.811+0000 7f9116ffd640 1 --2- 172.21.15.170:0/3673390289 >> v2:172.21.15.187:6816/101007897 conn(0x7f90f8076200 0x7f90f80786c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f91100072c0 tx=0x7f911000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:21.817 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:21.815+0000 7f910f7fe640 1 -- 172.21.15.170:0/3673390289 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f91180733e0 con 0x7f91181415e0 2024-05-08T04:08:22.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.043+0000 7f911d391640 1 -- 172.21.15.170:0/3673390289 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f9118069bd0 con 0x7f91181415e0 2024-05-08T04:08:22.048 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.046+0000 7f910f7fe640 1 -- 172.21.15.170:0/3673390289 <== mon.2 v2:172.21.15.170:3301/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+189369 (secure 0 0 0) 0x7f9108060cb0 con 0x7f91181415e0 2024-05-08T04:08:22.051 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:08:22.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.049+0000 7f911d391640 1 -- 172.21.15.170:0/3673390289 >> v2:172.21.15.187:6816/101007897 conn(0x7f90f8076200 msgr2=0x7f90f80786c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.049+0000 7f911d391640 1 --2- 172.21.15.170:0/3673390289 >> v2:172.21.15.187:6816/101007897 conn(0x7f90f8076200 0x7f90f80786c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f91100072c0 tx=0x7f911000b040 comp rx=0 tx=0).stop 2024-05-08T04:08:22.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.049+0000 7f911d391640 1 -- 172.21.15.170:0/3673390289 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f91181415e0 msgr2=0x7f91180af1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.049+0000 7f911d391640 1 --2- 172.21.15.170:0/3673390289 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f91181415e0 0x7f91180af1f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f910800e7f0 tx=0x7f910800ecb0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.049+0000 7f911d391640 1 -- 172.21.15.170:0/3673390289 shutdown_connections 2024-05-08T04:08:22.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.049+0000 7f911d391640 1 --2- 172.21.15.170:0/3673390289 >> v2:172.21.15.187:6816/101007897 conn(0x7f90f8076200 0x7f90f80786c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.049+0000 7f911d391640 1 --2- 172.21.15.170:0/3673390289 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f91180b0260 0x7f91181d7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.049+0000 7f911d391640 1 --2- 172.21.15.170:0/3673390289 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f91181415e0 0x7f91180af1f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.049+0000 7f911d391640 1 --2- 172.21.15.170:0/3673390289 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9118075ea0 0x7f91180ab9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.049+0000 7f911d391640 1 -- 172.21.15.170:0/3673390289 >> 172.21.15.170:0/3673390289 conn(0x7f91180b6020 msgr2=0x7f911813a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:22.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.049+0000 7f911d391640 1 -- 172.21.15.170:0/3673390289 shutdown_connections 2024-05-08T04:08:22.053 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.049+0000 7f911d391640 1 -- 172.21.15.170:0/3673390289 wait complete. 2024-05-08T04:08:22.062 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":5,"flags":0,"active_gid":4107,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":101007897}]},"active_addr":"172.21.15.187:6816/101007897","active_change":"2024-05-08T04:08:13.649286+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4108,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.187:0","nonce":3962183848}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.187:0","nonce":13679238}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.187:0","nonce":4206181845}]}]} 2024-05-08T04:08:22.068 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-08T04:08:22.068 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-08T04:08:22.068 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:08:22.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.167+0000 7f1267598640 1 Processor -- start 2024-05-08T04:08:22.169 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.167+0000 7f1267598640 1 -- start start 2024-05-08T04:08:22.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.168+0000 7f1267598640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f126006dca0 0x7f12601407b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.168+0000 7f1267598640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f12600b0500 0x7f126014e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.170 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.168+0000 7f1267598640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12600af1a0 con 0x7f1260140cf0 2024-05-08T04:08:22.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.168+0000 7f1267598640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7f126006e080 con 0x7f126006dca0 2024-05-08T04:08:22.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.168+0000 7f1267598640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f126006d810 con 0x7f12600b0500 2024-05-08T04:08:22.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.168+0000 7f1265334640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f126006dca0 0x7f12601407b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.168+0000 7f1264b33640 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f1260140cf0 0x7f12600afdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:55892/0 (socket says 172.21.15.170:55892) 2024-05-08T04:08:22.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.168+0000 7f1264b33640 1 -- 172.21.15.170:0/133276492 learned_addr learned my addr 172.21.15.170:0/133276492 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-05-08T04:08:22.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.168+0000 7f1265b35640 1 --2- 172.21.15.170:0/133276492 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f12600b0500 0x7f126014e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.168+0000 7f1265334640 1 -- 172.21.15.170:0/133276492 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f12600b0500 msgr2=0x7f126014e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:22.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.168+0000 7f1265334640 1 --2- 172.21.15.170:0/133276492 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f12600b0500 0x7f126014e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.168+0000 7f1265334640 1 -- 172.21.15.170:0/133276492 >> v1:172.21.15.170:6789/0 conn(0x7f1260140cf0 legacy=0x7f12600afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.168+0000 7f1265334640 1 -- 172.21.15.170:0/133276492 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12600af410 con 0x7f126006dca0 2024-05-08T04:08:22.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.168+0000 7f1265b35640 1 --2- 172.21.15.170:0/133276492 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f12600b0500 0x7f126014e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:08:22.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.169+0000 7f1265334640 1 --2- 172.21.15.170:0/133276492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f126006dca0 0x7f12601407b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f125c009d90 tx=0x7f125c02bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e5a6558c839538 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.169+0000 7f1257fff640 1 -- 172.21.15.170:0/133276492 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c039070 con 0x7f126006dca0 2024-05-08T04:08:22.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.169+0000 7f1257fff640 1 -- 172.21.15.170:0/133276492 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f125c02cb30 con 0x7f126006dca0 2024-05-08T04:08:22.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.169+0000 7f1257fff640 1 -- 172.21.15.170:0/133276492 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c034440 con 0x7f126006dca0 2024-05-08T04:08:22.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.169+0000 7f1267598640 1 -- 172.21.15.170:0/133276492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f126006dca0 msgr2=0x7f12601407b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.169+0000 7f1267598640 1 --2- 172.21.15.170:0/133276492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f126006dca0 0x7f12601407b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f125c009d90 tx=0x7f125c02bde0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.169+0000 7f1267598640 1 -- 172.21.15.170:0/133276492 shutdown_connections 2024-05-08T04:08:22.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.169+0000 7f1267598640 1 --2- 172.21.15.170:0/133276492 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f12600b0500 0x7f126014e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.169+0000 7f1267598640 1 --2- 172.21.15.170:0/133276492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f126006dca0 0x7f12601407b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.169+0000 7f1267598640 1 -- 172.21.15.170:0/133276492 >> 172.21.15.170:0/133276492 conn(0x7f12600b3de0 msgr2=0x7f12600b41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:22.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.170+0000 7f1267598640 1 -- 172.21.15.170:0/133276492 shutdown_connections 2024-05-08T04:08:22.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.170+0000 7f1267598640 1 -- 172.21.15.170:0/133276492 wait complete. 2024-05-08T04:08:22.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.171+0000 7f1267598640 1 Processor -- start 2024-05-08T04:08:22.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.171+0000 7f1267598640 1 -- start start 2024-05-08T04:08:22.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.171+0000 7f1267598640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f126006dca0 0x7f1260073ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.171+0000 7f1267598640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f12600b0500 0x7f12601361c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.171+0000 7f1267598640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1260140cf0 0x7f1260136700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.171+0000 7f1267598640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f12600af9d0 con 0x7f126006dca0 2024-05-08T04:08:22.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.171+0000 7f1267598640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1260141160 con 0x7f1260140cf0 2024-05-08T04:08:22.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.171+0000 7f1267598640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f12600b1530 con 0x7f12600b0500 2024-05-08T04:08:22.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.171+0000 7f1265334640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f126006dca0 0x7f1260073ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.171+0000 7f1265334640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f126006dca0 0x7f1260073ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:48988/0 (socket says 172.21.15.170:48988) 2024-05-08T04:08:22.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.171+0000 7f1265334640 1 -- 172.21.15.170:0/4129779527 learned_addr learned my addr 172.21.15.170:0/4129779527 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:22.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.171+0000 7f1264b33640 1 --2- 172.21.15.170:0/4129779527 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f12600b0500 0x7f12601361c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.171+0000 7f1265b35640 1 --2- 172.21.15.170:0/4129779527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1260140cf0 0x7f1260136700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.171+0000 7f1265334640 1 -- 172.21.15.170:0/4129779527 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f12600b0500 msgr2=0x7f12601361c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.171+0000 7f1265334640 1 --2- 172.21.15.170:0/4129779527 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f12600b0500 0x7f12601361c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.171+0000 7f1265334640 1 -- 172.21.15.170:0/4129779527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1260140cf0 msgr2=0x7f1260136700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.171+0000 7f1265334640 1 --2- 172.21.15.170:0/4129779527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1260140cf0 0x7f1260136700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.171+0000 7f1265334640 1 -- 172.21.15.170:0/4129779527 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12601cd760 con 0x7f126006dca0 2024-05-08T04:08:22.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.171+0000 7f1265b35640 1 --2- 172.21.15.170:0/4129779527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1260140cf0 0x7f1260136700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:08:22.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.172+0000 7f1265334640 1 --2- 172.21.15.170:0/4129779527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f126006dca0 0x7f1260073ff0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f125c009eb0 tx=0x7f125c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.172+0000 7f1255ffb640 1 -- 172.21.15.170:0/4129779527 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c039050 con 0x7f126006dca0 2024-05-08T04:08:22.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.172+0000 7f1255ffb640 1 -- 172.21.15.170:0/4129779527 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f125c045070 con 0x7f126006dca0 2024-05-08T04:08:22.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.172+0000 7f1255ffb640 1 -- 172.21.15.170:0/4129779527 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f125c03e850 con 0x7f126006dca0 2024-05-08T04:08:22.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.172+0000 7f1267598640 1 -- 172.21.15.170:0/4129779527 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12601cda40 con 0x7f126006dca0 2024-05-08T04:08:22.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.172+0000 7f1267598640 1 -- 172.21.15.170:0/4129779527 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f126013d670 con 0x7f126006dca0 2024-05-08T04:08:22.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.173+0000 7f1267598640 1 -- 172.21.15.170:0/4129779527 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f126006b400 con 0x7f126006dca0 2024-05-08T04:08:22.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.173+0000 7f1255ffb640 1 -- 172.21.15.170:0/4129779527 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f125c03e9f0 con 0x7f126006dca0 2024-05-08T04:08:22.180 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.174+0000 7f1255ffb640 1 --2- 172.21.15.170:0/4129779527 >> v2:172.21.15.187:6816/101007897 conn(0x7f1244076290 0x7f1244078750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.174+0000 7f1255ffb640 1 -- 172.21.15.170:0/4129779527 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4090+0+0 (secure 0 0 0) 0x7f125c0b9ca0 con 0x7f126006dca0 2024-05-08T04:08:22.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.174+0000 7f1264b33640 1 --2- 172.21.15.170:0/4129779527 >> v2:172.21.15.187:6816/101007897 conn(0x7f1244076290 0x7f1244078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.175+0000 7f1264b33640 1 --2- 172.21.15.170:0/4129779527 >> v2:172.21.15.187:6816/101007897 conn(0x7f1244076290 0x7f1244078750 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f12601d0a50 tx=0x7f1250009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.178+0000 7f1255ffb640 1 -- 172.21.15.170:0/4129779527 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f12600af410 con 0x7f126006dca0 2024-05-08T04:08:22.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.359+0000 7f1267598640 1 -- 172.21.15.170:0/4129779527 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f126006e8e0 con 0x7f126006dca0 2024-05-08T04:08:22.362 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.360+0000 7f1255ffb640 1 -- 172.21.15.170:0/4129779527 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12816 (secure 0 0 0) 0x7f125c083ce0 con 0x7f126006dca0 2024-05-08T04:08:22.362 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:08:22.362 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":17,"fsid":"e5497bb3-afa8-4f24-8ef4-42911ffcc962","created":"2024-05-08T04:08:09.270498+0000","modified":"2024-05-08T04:08:21.685301+0000","last_up_change":"2024-05-08T04:08:18.673075+0000","last_in_change":"2024-05-08T04:08:11.562526+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:08:19.670334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-08T04:08:20.197207+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6bca968b-d825-417a-b907-5312aad8dbcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1037021465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6801","nonce":1037021465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6803","nonce":1037021465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1037021465}]},"public_addr":"172.21.15.170:6800/1037021465","cluster_addr":"172.21.15.170:6801/1037021465","heartbeat_back_addr":"172.21.15.170:6803/1037021465","heartbeat_front_addr":"172.21.15.170:6802/1037021465","state":["exists","up"]},{"osd":1,"uuid":"66600f89-4532-43b1-85d3-6d8c736cfaf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3364584304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6809","nonce":3364584304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6811","nonce":3364584304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3364584304}]},"public_addr":"172.21.15.170:6808/3364584304","cluster_addr":"172.21.15.170:6809/3364584304","heartbeat_back_addr":"172.21.15.170:6811/3364584304","heartbeat_front_addr":"172.21.15.170:6810/3364584304","state":["exists","up"]},{"osd":2,"uuid":"a11608b2-0871-497c-aa38-264baea6bf39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":3333432230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6805","nonce":3333432230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6807","nonce":3333432230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":3333432230}]},"public_addr":"172.21.15.170:6804/3333432230","cluster_addr":"172.21.15.170:6805/3333432230","heartbeat_back_addr":"172.21.15.170:6807/3333432230","heartbeat_front_addr":"172.21.15.170:6806/3333432230","state":["exists","up"]},{"osd":3,"uuid":"8caae6f8-790c-426c-af8a-3a9ef54eae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":153555120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6813","nonce":153555120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6815","nonce":153555120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":153555120}]},"public_addr":"172.21.15.170:6812/153555120","cluster_addr":"172.21.15.170:6813/153555120","heartbeat_back_addr":"172.21.15.170:6815/153555120","heartbeat_front_addr":"172.21.15.170:6814/153555120","state":["exists","up"]},{"osd":4,"uuid":"0e804208-3607-4d71-b771-f79fb27e7968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1425849822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1425849822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1425849822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":1425849822}]},"public_addr":"172.21.15.187:6808/1425849822","cluster_addr":"172.21.15.187:6811/1425849822","heartbeat_back_addr":"172.21.15.187:6815/1425849822","heartbeat_front_addr":"172.21.15.187:6813/1425849822","state":["exists","up"]},{"osd":5,"uuid":"f88f88db-216e-4acc-ba5a-89d91b98aeb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":3273856499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":3273856499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6803","nonce":3273856499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":3273856499}]},"public_addr":"172.21.15.187:6800/3273856499","cluster_addr":"172.21.15.187:6801/3273856499","heartbeat_back_addr":"172.21.15.187:6803/3273856499","heartbeat_front_addr":"172.21.15.187:6802/3273856499","state":["exists","up"]},{"osd":6,"uuid":"6b6840fc-1e34-4cc8-9da6-8a57e69f319b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":2911331731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2911331731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":2911331731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":2911331731}]},"public_addr":"172.21.15.187:6809/2911331731","cluster_addr":"172.21.15.187:6810/2911331731","heartbeat_back_addr":"172.21.15.187:6814/2911331731","heartbeat_front_addr":"172.21.15.187:6812/2911331731","state":["exists","up"]},{"osd":7,"uuid":"5dbcf804-181b-4d5b-9a40-f99ddaee2701","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2815417961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":2815417961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6807","nonce":2815417961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2815417961}]},"public_addr":"172.21.15.187:6804/2815417961","cluster_addr":"172.21.15.187:6805/2815417961","heartbeat_back_addr":"172.21.15.187:6807/2815417961","heartbeat_front_addr":"172.21.15.187:6806/2815417961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:08:16.738630+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:08:16.681089+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:08:16.630258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:08:16.670263+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:08:16.737547+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:08:16.673870+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:08:16.706810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:08:16.658743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:08:22.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.362+0000 7f1267598640 1 -- 172.21.15.170:0/4129779527 >> v2:172.21.15.187:6816/101007897 conn(0x7f1244076290 msgr2=0x7f1244078750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.362+0000 7f1267598640 1 --2- 172.21.15.170:0/4129779527 >> v2:172.21.15.187:6816/101007897 conn(0x7f1244076290 0x7f1244078750 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f12601d0a50 tx=0x7f1250009040 comp rx=0 tx=0).stop 2024-05-08T04:08:22.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.362+0000 7f1267598640 1 -- 172.21.15.170:0/4129779527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f126006dca0 msgr2=0x7f1260073ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.362+0000 7f1267598640 1 --2- 172.21.15.170:0/4129779527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f126006dca0 0x7f1260073ff0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f125c009eb0 tx=0x7f125c004a40 comp rx=0 tx=0).stop 2024-05-08T04:08:22.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.362+0000 7f1267598640 1 -- 172.21.15.170:0/4129779527 shutdown_connections 2024-05-08T04:08:22.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.362+0000 7f1267598640 1 --2- 172.21.15.170:0/4129779527 >> v2:172.21.15.187:6816/101007897 conn(0x7f1244076290 0x7f1244078750 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.362+0000 7f1267598640 1 --2- 172.21.15.170:0/4129779527 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1260140cf0 0x7f1260136700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.362+0000 7f1267598640 1 --2- 172.21.15.170:0/4129779527 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f12600b0500 0x7f12601361c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.362+0000 7f1267598640 1 --2- 172.21.15.170:0/4129779527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f126006dca0 0x7f1260073ff0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.362+0000 7f1267598640 1 -- 172.21.15.170:0/4129779527 >> 172.21.15.170:0/4129779527 conn(0x7f12600b3de0 msgr2=0x7f1260068180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:22.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.362+0000 7f1267598640 1 -- 172.21.15.170:0/4129779527 shutdown_connections 2024-05-08T04:08:22.366 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.362+0000 7f1267598640 1 -- 172.21.15.170:0/4129779527 wait complete. 2024-05-08T04:08:22.376 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-08T04:08:22.376 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:08:22.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.473+0000 7fe9b8de5640 1 Processor -- start 2024-05-08T04:08:22.475 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.473+0000 7fe9b8de5640 1 -- start start 2024-05-08T04:08:22.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.474+0000 7fe9b8de5640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe9b40b43a0 0x7fe9b406de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.476 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.474+0000 7fe9b8de5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe9b4140f00 0x7fe9b414e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.474+0000 7fe9b8de5640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9b40af3f0 con 0x7fe9b40afdd0 2024-05-08T04:08:22.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.474+0000 7fe9b8de5640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe9b406d680 con 0x7fe9b40b43a0 2024-05-08T04:08:22.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.474+0000 7fe9b8de5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fe9b40af6d0 con 0x7fe9b4140f00 2024-05-08T04:08:22.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.474+0000 7fe9b2d9d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe9b4140f00 0x7fe9b414e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.474+0000 7fe9b2d9d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe9b4140f00 0x7fe9b414e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:54234/0 (socket says 172.21.15.170:54234) 2024-05-08T04:08:22.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.474+0000 7fe9b2d9d640 1 -- 172.21.15.170:0/1072951222 learned_addr learned my addr 172.21.15.170:0/1072951222 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:22.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.474+0000 7fe9b259c640 1 --2- 172.21.15.170:0/1072951222 >> v2:172.21.15.187:3300/0 conn(0x7fe9b40b43a0 0x7fe9b406de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.474+0000 7fe9b2d9d640 1 -- 172.21.15.170:0/1072951222 >> v2:172.21.15.187:3300/0 conn(0x7fe9b40b43a0 msgr2=0x7fe9b406de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:22.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.474+0000 7fe9b2d9d640 1 --2- 172.21.15.170:0/1072951222 >> v2:172.21.15.187:3300/0 conn(0x7fe9b40b43a0 0x7fe9b406de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.474+0000 7fe9b2d9d640 1 -- 172.21.15.170:0/1072951222 >> v1:172.21.15.187:6789/0 conn(0x7fe9b40afdd0 legacy=0x7fe9b41407f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.477 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.474+0000 7fe9b2d9d640 1 -- 172.21.15.170:0/1072951222 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b40af9b0 con 0x7fe9b4140f00 2024-05-08T04:08:22.478 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.475+0000 7fe9b2d9d640 1 --2- 172.21.15.170:0/1072951222 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe9b4140f00 0x7fe9b414e0a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac009870 tx=0x7fe9ac02b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cf019e4faa97398 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.475+0000 7fe9b159a640 1 -- 172.21.15.170:0/1072951222 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac0317c0 con 0x7fe9b4140f00 2024-05-08T04:08:22.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.475+0000 7fe9b159a640 1 -- 172.21.15.170:0/1072951222 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9ac037070 con 0x7fe9b4140f00 2024-05-08T04:08:22.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.475+0000 7fe9b159a640 1 -- 172.21.15.170:0/1072951222 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac031b80 con 0x7fe9b4140f00 2024-05-08T04:08:22.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.476+0000 7fe9b8de5640 1 -- 172.21.15.170:0/1072951222 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe9b4140f00 msgr2=0x7fe9b414e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.476+0000 7fe9b8de5640 1 --2- 172.21.15.170:0/1072951222 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe9b4140f00 0x7fe9b414e0a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac009870 tx=0x7fe9ac02b860 comp rx=0 tx=0).stop 2024-05-08T04:08:22.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.476+0000 7fe9b8de5640 1 -- 172.21.15.170:0/1072951222 shutdown_connections 2024-05-08T04:08:22.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.476+0000 7fe9b8de5640 1 --2- 172.21.15.170:0/1072951222 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe9b4140f00 0x7fe9b414e0a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.476+0000 7fe9b8de5640 1 --2- 172.21.15.170:0/1072951222 >> v2:172.21.15.187:3300/0 conn(0x7fe9b40b43a0 0x7fe9b406de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.476+0000 7fe9b8de5640 1 -- 172.21.15.170:0/1072951222 >> 172.21.15.170:0/1072951222 conn(0x7fe9b40b74f0 msgr2=0x7fe9b40b39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:22.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.476+0000 7fe9b8de5640 1 -- 172.21.15.170:0/1072951222 shutdown_connections 2024-05-08T04:08:22.479 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.476+0000 7fe9b8de5640 1 -- 172.21.15.170:0/1072951222 wait complete. 2024-05-08T04:08:22.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.477+0000 7fe9b8de5640 1 Processor -- start 2024-05-08T04:08:22.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.477+0000 7fe9b8de5640 1 -- start start 2024-05-08T04:08:22.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.478+0000 7fe9b8de5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe9b40afdd0 0x7fe9b41dc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.478+0000 7fe9b8de5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9b40b43a0 0x7fe9b4072c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.478+0000 7fe9b8de5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9b4140f00 0x7fe9b41dcc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.478+0000 7fe9b8de5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b40b1600 con 0x7fe9b40b43a0 2024-05-08T04:08:22.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.478+0000 7fe9b8de5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b40b1480 con 0x7fe9b4140f00 2024-05-08T04:08:22.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.478+0000 7fe9b8de5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fe9b40b0240 con 0x7fe9b40afdd0 2024-05-08T04:08:22.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.478+0000 7fe9b259c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe9b40afdd0 0x7fe9b41dc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.478+0000 7fe9b1d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9b40b43a0 0x7fe9b4072c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.478+0000 7fe9b259c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe9b40afdd0 0x7fe9b41dc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:54242/0 (socket says 172.21.15.170:54242) 2024-05-08T04:08:22.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.478+0000 7fe9b259c640 1 -- 172.21.15.170:0/2633561553 learned_addr learned my addr 172.21.15.170:0/2633561553 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:22.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.478+0000 7fe9b2d9d640 1 --2- 172.21.15.170:0/2633561553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9b4140f00 0x7fe9b41dcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.478+0000 7fe9b1d9b640 1 -- 172.21.15.170:0/2633561553 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe9b40afdd0 msgr2=0x7fe9b41dc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.478+0000 7fe9b1d9b640 1 --2- 172.21.15.170:0/2633561553 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe9b40afdd0 0x7fe9b41dc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.478+0000 7fe9b1d9b640 1 -- 172.21.15.170:0/2633561553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9b4140f00 msgr2=0x7fe9b41dcc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.482 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.478+0000 7fe9b1d9b640 1 --2- 172.21.15.170:0/2633561553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9b4140f00 0x7fe9b41dcc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.478+0000 7fe9b1d9b640 1 -- 172.21.15.170:0/2633561553 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b4073150 con 0x7fe9b40b43a0 2024-05-08T04:08:22.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.478+0000 7fe9b1d9b640 1 --2- 172.21.15.170:0/2633561553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9b40b43a0 0x7fe9b4072c10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800c3c0 tx=0x7fe9a800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.478+0000 7fe9a2ffd640 1 -- 172.21.15.170:0/2633561553 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8017070 con 0x7fe9b40b43a0 2024-05-08T04:08:22.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.478+0000 7fe9a2ffd640 1 -- 172.21.15.170:0/2633561553 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9a80052d0 con 0x7fe9b40b43a0 2024-05-08T04:08:22.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.478+0000 7fe9b8de5640 1 -- 172.21.15.170:0/2633561553 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9b41dd170 con 0x7fe9b40b43a0 2024-05-08T04:08:22.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.478+0000 7fe9a2ffd640 1 -- 172.21.15.170:0/2633561553 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8003830 con 0x7fe9b40b43a0 2024-05-08T04:08:22.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.478+0000 7fe9b8de5640 1 -- 172.21.15.170:0/2633561553 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9b4137d50 con 0x7fe9b40b43a0 2024-05-08T04:08:22.483 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.479+0000 7fe9b8de5640 1 -- 172.21.15.170:0/2633561553 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9b406b400 con 0x7fe9b40b43a0 2024-05-08T04:08:22.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.480+0000 7fe9a2ffd640 1 -- 172.21.15.170:0/2633561553 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7fe9a801a070 con 0x7fe9b40b43a0 2024-05-08T04:08:22.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.480+0000 7fe9a2ffd640 1 --2- 172.21.15.170:0/2633561553 >> v2:172.21.15.187:6816/101007897 conn(0x7fe994076290 0x7fe994078750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.480+0000 7fe9a2ffd640 1 -- 172.21.15.170:0/2633561553 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4090+0+0 (secure 0 0 0) 0x7fe9a80961d0 con 0x7fe9b40b43a0 2024-05-08T04:08:22.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.481+0000 7fe9b259c640 1 --2- 172.21.15.170:0/2633561553 >> v2:172.21.15.187:6816/101007897 conn(0x7fe994076290 0x7fe994078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.481+0000 7fe9b259c640 1 --2- 172.21.15.170:0/2633561553 >> v2:172.21.15.187:6816/101007897 conn(0x7fe994076290 0x7fe994078750 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4009a40 tx=0x7fe9a400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.487 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.485+0000 7fe9a2ffd640 1 -- 172.21.15.170:0/2633561553 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe9a805fb40 con 0x7fe9b40b43a0 2024-05-08T04:08:22.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.667+0000 7fe9b8de5640 1 -- 172.21.15.170:0/2633561553 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9b40b0f90 con 0x7fe9b40b43a0 2024-05-08T04:08:22.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.668+0000 7fe9a2ffd640 1 -- 172.21.15.170:0/2633561553 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12816 (secure 0 0 0) 0x7fe9a8063e40 con 0x7fe9b40b43a0 2024-05-08T04:08:22.670 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:08:22.670 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":17,"fsid":"e5497bb3-afa8-4f24-8ef4-42911ffcc962","created":"2024-05-08T04:08:09.270498+0000","modified":"2024-05-08T04:08:21.685301+0000","last_up_change":"2024-05-08T04:08:18.673075+0000","last_in_change":"2024-05-08T04:08:11.562526+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:08:19.670334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-08T04:08:20.197207+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6bca968b-d825-417a-b907-5312aad8dbcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1037021465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6801","nonce":1037021465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6803","nonce":1037021465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1037021465}]},"public_addr":"172.21.15.170:6800/1037021465","cluster_addr":"172.21.15.170:6801/1037021465","heartbeat_back_addr":"172.21.15.170:6803/1037021465","heartbeat_front_addr":"172.21.15.170:6802/1037021465","state":["exists","up"]},{"osd":1,"uuid":"66600f89-4532-43b1-85d3-6d8c736cfaf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3364584304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6809","nonce":3364584304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6811","nonce":3364584304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3364584304}]},"public_addr":"172.21.15.170:6808/3364584304","cluster_addr":"172.21.15.170:6809/3364584304","heartbeat_back_addr":"172.21.15.170:6811/3364584304","heartbeat_front_addr":"172.21.15.170:6810/3364584304","state":["exists","up"]},{"osd":2,"uuid":"a11608b2-0871-497c-aa38-264baea6bf39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":3333432230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6805","nonce":3333432230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6807","nonce":3333432230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":3333432230}]},"public_addr":"172.21.15.170:6804/3333432230","cluster_addr":"172.21.15.170:6805/3333432230","heartbeat_back_addr":"172.21.15.170:6807/3333432230","heartbeat_front_addr":"172.21.15.170:6806/3333432230","state":["exists","up"]},{"osd":3,"uuid":"8caae6f8-790c-426c-af8a-3a9ef54eae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":153555120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6813","nonce":153555120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6815","nonce":153555120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":153555120}]},"public_addr":"172.21.15.170:6812/153555120","cluster_addr":"172.21.15.170:6813/153555120","heartbeat_back_addr":"172.21.15.170:6815/153555120","heartbeat_front_addr":"172.21.15.170:6814/153555120","state":["exists","up"]},{"osd":4,"uuid":"0e804208-3607-4d71-b771-f79fb27e7968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1425849822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1425849822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1425849822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":1425849822}]},"public_addr":"172.21.15.187:6808/1425849822","cluster_addr":"172.21.15.187:6811/1425849822","heartbeat_back_addr":"172.21.15.187:6815/1425849822","heartbeat_front_addr":"172.21.15.187:6813/1425849822","state":["exists","up"]},{"osd":5,"uuid":"f88f88db-216e-4acc-ba5a-89d91b98aeb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":3273856499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":3273856499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6803","nonce":3273856499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":3273856499}]},"public_addr":"172.21.15.187:6800/3273856499","cluster_addr":"172.21.15.187:6801/3273856499","heartbeat_back_addr":"172.21.15.187:6803/3273856499","heartbeat_front_addr":"172.21.15.187:6802/3273856499","state":["exists","up"]},{"osd":6,"uuid":"6b6840fc-1e34-4cc8-9da6-8a57e69f319b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":2911331731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2911331731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":2911331731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":2911331731}]},"public_addr":"172.21.15.187:6809/2911331731","cluster_addr":"172.21.15.187:6810/2911331731","heartbeat_back_addr":"172.21.15.187:6814/2911331731","heartbeat_front_addr":"172.21.15.187:6812/2911331731","state":["exists","up"]},{"osd":7,"uuid":"5dbcf804-181b-4d5b-9a40-f99ddaee2701","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2815417961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":2815417961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6807","nonce":2815417961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2815417961}]},"public_addr":"172.21.15.187:6804/2815417961","cluster_addr":"172.21.15.187:6805/2815417961","heartbeat_back_addr":"172.21.15.187:6807/2815417961","heartbeat_front_addr":"172.21.15.187:6806/2815417961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:08:16.738630+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:08:16.681089+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:08:16.630258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:08:16.670263+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:08:16.737547+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:08:16.673870+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:08:16.706810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:08:16.658743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:08:22.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.670+0000 7fe9b8de5640 1 -- 172.21.15.170:0/2633561553 >> v2:172.21.15.187:6816/101007897 conn(0x7fe994076290 msgr2=0x7fe994078750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.670+0000 7fe9b8de5640 1 --2- 172.21.15.170:0/2633561553 >> v2:172.21.15.187:6816/101007897 conn(0x7fe994076290 0x7fe994078750 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4009a40 tx=0x7fe9a400b040 comp rx=0 tx=0).stop 2024-05-08T04:08:22.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.670+0000 7fe9b8de5640 1 -- 172.21.15.170:0/2633561553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9b40b43a0 msgr2=0x7fe9b4072c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.670+0000 7fe9b8de5640 1 --2- 172.21.15.170:0/2633561553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9b40b43a0 0x7fe9b4072c10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800c3c0 tx=0x7fe9a800c880 comp rx=0 tx=0).stop 2024-05-08T04:08:22.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.670+0000 7fe9b8de5640 1 -- 172.21.15.170:0/2633561553 shutdown_connections 2024-05-08T04:08:22.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.670+0000 7fe9b8de5640 1 --2- 172.21.15.170:0/2633561553 >> v2:172.21.15.187:6816/101007897 conn(0x7fe994076290 0x7fe994078750 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.670+0000 7fe9b8de5640 1 --2- 172.21.15.170:0/2633561553 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe9b4140f00 0x7fe9b41dcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.672 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.670+0000 7fe9b8de5640 1 --2- 172.21.15.170:0/2633561553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9b40b43a0 0x7fe9b4072c10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.670+0000 7fe9b8de5640 1 --2- 172.21.15.170:0/2633561553 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe9b40afdd0 0x7fe9b41dc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.670+0000 7fe9b8de5640 1 -- 172.21.15.170:0/2633561553 >> 172.21.15.170:0/2633561553 conn(0x7fe9b40b74f0 msgr2=0x7fe9b413fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:22.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.670+0000 7fe9b8de5640 1 -- 172.21.15.170:0/2633561553 shutdown_connections 2024-05-08T04:08:22.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.670+0000 7fe9b8de5640 1 -- 172.21.15.170:0/2633561553 wait complete. 2024-05-08T04:08:22.683 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-08T04:08:22.684 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-08T04:08:22.684 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-08T04:08:22.684 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-08T04:08:22.685 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-08T04:08:22.685 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-08T04:08:22.686 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-08T04:08:22.686 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-08T04:08:22.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.855+0000 7fdc97fff640 1 Processor -- start 2024-05-08T04:08:22.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.855+0000 7fdc97fff640 1 -- start start 2024-05-08T04:08:22.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.856+0000 7f4acef5e640 1 Processor -- start 2024-05-08T04:08:22.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.856+0000 7fdc97fff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdc9806fc80 0x7fdc980af970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.856+0000 7fdc97fff640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc98071910 con 0x7fdc9814bd60 2024-05-08T04:08:22.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.856+0000 7fdc97fff640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc98070060 con 0x7fdc980739c0 2024-05-08T04:08:22.859 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.856+0000 7fdc97fff640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fdc98071bf0 con 0x7fdc9806fc80 2024-05-08T04:08:22.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.856+0000 7f4acef5e640 1 -- start start 2024-05-08T04:08:22.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.857+0000 7f4acef5e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4ac8070de0 0x7f4ac813f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.857+0000 7f4acef5e640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4ac813ff00 0x7f4ac8140390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.857+0000 7f4acef5e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ac8073170 con 0x7f4ac80700a0 2024-05-08T04:08:22.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.857+0000 7f4acef5e640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7f4ac8072080 con 0x7f4ac813ff00 2024-05-08T04:08:22.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.857+0000 7f4acef5e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f4ac8072360 con 0x7f4ac8070de0 2024-05-08T04:08:22.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.858+0000 7f4ace75d640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4ac813ff00 0x7f4ac8140390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.858+0000 7f4ace75d640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4ac813ff00 0x7f4ac8140390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49014/0 (socket says 172.21.15.170:49014) 2024-05-08T04:08:22.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.858+0000 7f4ace75d640 1 -- 172.21.15.170:0/2906882768 learned_addr learned my addr 172.21.15.170:0/2906882768 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:22.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.858+0000 7f66cef2d640 1 Processor -- start 2024-05-08T04:08:22.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.856+0000 7fdc977fe640 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fdc9814bd60 0x7fdc98152450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:55906/0 (socket says 172.21.15.170:55906) 2024-05-08T04:08:22.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.858+0000 7f4acd75b640 1 --2- 172.21.15.170:0/2906882768 >> v2:172.21.15.187:3300/0 conn(0x7f4ac8070de0 0x7f4ac813f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.856+0000 7fdc977fe640 1 -- 172.21.15.170:0/140552821 learned_addr learned my addr 172.21.15.170:0/140552821 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-05-08T04:08:22.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.857+0000 7fdc95ffb640 1 -- 172.21.15.170:0/140552821 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2461694988 0 0) 0x7fdc90001490 con 0x7fdc9814bd60 2024-05-08T04:08:22.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.857+0000 7fdc95ffb640 1 -- 172.21.15.170:0/140552821 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc80003520 con 0x7fdc9814bd60 2024-05-08T04:08:22.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.857+0000 7fdc95ffb640 1 -- 172.21.15.170:0/140552821 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2793322662 0 0) 0x7fdc80003520 con 0x7fdc9814bd60 2024-05-08T04:08:22.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.857+0000 7fdc95ffb640 1 -- 172.21.15.170:0/140552821 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdc9806fc80 msgr2=0x7fdc980af970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T04:08:22.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.857+0000 7fdc95ffb640 1 --2- 172.21.15.170:0/140552821 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdc9806fc80 0x7fdc980af970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.857+0000 7fdc95ffb640 1 -- 172.21.15.170:0/140552821 >> v1:172.21.15.187:6789/0 conn(0x7fdc980739c0 legacy=0x7fdc9814b650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T04:08:22.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.857+0000 7fdc95ffb640 1 -- 172.21.15.170:0/140552821 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc98071ed0 con 0x7fdc9814bd60 2024-05-08T04:08:22.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.857+0000 7fdc95ffb640 1 -- 172.21.15.170:0/140552821 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1210551688 0 0) 0x7fdc90001c40 con 0x7fdc9814bd60 2024-05-08T04:08:22.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.857+0000 7fdc95ffb640 1 -- 172.21.15.170:0/140552821 >> v1:172.21.15.170:6789/0 conn(0x7fdc9814bd60 legacy=0x7fdc98152450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.857+0000 7fdc95ffb640 1 --2- 172.21.15.170:0/140552821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc80003a30 0x7fdc80003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.857+0000 7fdc95ffb640 1 --2- 172.21.15.170:0/140552821 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdc80004800 0x7fdc80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.857+0000 7fdc95ffb640 1 --2- 172.21.15.170:0/140552821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc80005210 0x7fdc8000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.857+0000 7fdc95ffb640 1 -- 172.21.15.170:0/140552821 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdc98071bf0 con 0x7fdc80003a30 2024-05-08T04:08:22.862 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.857+0000 7fdc95ffb640 1 -- 172.21.15.170:0/140552821 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8000c140 con 0x7fdc80005210 2024-05-08T04:08:22.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.857+0000 7fdc95ffb640 1 -- 172.21.15.170:0/140552821 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fdc8000c340 con 0x7fdc80004800 2024-05-08T04:08:22.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.858+0000 7fdc96ffd640 1 --2- 172.21.15.170:0/140552821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc80003a30 0x7fdc80003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.858+0000 7fdc977fe640 1 --2- 172.21.15.170:0/140552821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc80005210 0x7fdc8000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.858+0000 7fdc967fc640 1 --2- 172.21.15.170:0/140552821 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdc80004800 0x7fdc80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.858+0000 7fdc96ffd640 1 -- 172.21.15.170:0/140552821 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdc80004800 msgr2=0x7fdc80004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:22.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.858+0000 7fdc96ffd640 1 --2- 172.21.15.170:0/140552821 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdc80004800 0x7fdc80004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.858+0000 7fdc96ffd640 1 -- 172.21.15.170:0/140552821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc80005210 msgr2=0x7fdc8000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:22.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.858+0000 7fdc96ffd640 1 --2- 172.21.15.170:0/140552821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc80005210 0x7fdc8000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.858+0000 7fdc96ffd640 1 -- 172.21.15.170:0/140552821 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc98071ed0 con 0x7fdc80003a30 2024-05-08T04:08:22.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.858+0000 7f4accf5a640 1 -- 172.21.15.170:0/2906882768 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1974714831 0 0) 0x7f4ac4001110 con 0x7f4ac80700a0 2024-05-08T04:08:22.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.858+0000 7f4accf5a640 1 -- 172.21.15.170:0/2906882768 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ab4003520 con 0x7f4ac80700a0 2024-05-08T04:08:22.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.858+0000 7f4acd75b640 1 -- 172.21.15.170:0/2906882768 >> v2:172.21.15.170:3300/0 conn(0x7f4ac813ff00 msgr2=0x7f4ac8140390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:22.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.858+0000 7f4acd75b640 1 --2- 172.21.15.170:0/2906882768 >> v2:172.21.15.170:3300/0 conn(0x7f4ac813ff00 0x7f4ac8140390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.858+0000 7f4acd75b640 1 -- 172.21.15.170:0/2906882768 >> v1:172.21.15.187:6789/0 conn(0x7f4ac80700a0 legacy=0x7f4ac8072a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.858+0000 7f4acd75b640 1 -- 172.21.15.170:0/2906882768 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac8072640 con 0x7f4ac8070de0 2024-05-08T04:08:22.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.858+0000 7f66cef2d640 1 -- start start 2024-05-08T04:08:22.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.859+0000 7f66cef2d640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f66c8073990 0x7f66c8071100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.859+0000 7f66cef2d640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66c8142cf0 con 0x7f66c8070320 2024-05-08T04:08:22.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.859+0000 7f66cef2d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66c8072500 con 0x7f66c8070700 2024-05-08T04:08:22.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.859+0000 7f66cef2d640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7f66c80727e0 con 0x7f66c8073990 2024-05-08T04:08:22.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.859+0000 7f66cccc9640 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f66c8070320 0x7f66c813cc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:55914/0 (socket says 172.21.15.170:55914) 2024-05-08T04:08:22.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.859+0000 7f66cccc9640 1 -- 172.21.15.170:0/338644855 learned_addr learned my addr 172.21.15.170:0/338644855 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-05-08T04:08:22.864 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.859+0000 7f66cd4ca640 1 --2- 172.21.15.170:0/338644855 >> v2:172.21.15.170:3300/0 conn(0x7f66c8073990 0x7f66c8071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.859+0000 7f66cd4ca640 1 -- 172.21.15.170:0/338644855 >> v1:172.21.15.187:6789/0 conn(0x7f66c8070700 legacy=0x7f66c8073280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.859+0000 7f66cd4ca640 1 -- 172.21.15.170:0/338644855 >> v1:172.21.15.170:6789/0 conn(0x7f66c8070320 legacy=0x7f66c813cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.859+0000 7f66cd4ca640 1 -- 172.21.15.170:0/338644855 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c8072ac0 con 0x7f66c8073990 2024-05-08T04:08:22.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.860+0000 7f4acd75b640 1 --2- 172.21.15.170:0/2906882768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ac8070de0 0x7f4ac813f9c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8074430 tx=0x7f4ac002e060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16153bc2bfb867ac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.860+0000 7f4accf5a640 1 -- 172.21.15.170:0/2906882768 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac002ea00 con 0x7f4ac8070de0 2024-05-08T04:08:22.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.860+0000 7feeb36ed640 1 Processor -- start 2024-05-08T04:08:22.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.860+0000 7feeb36ed640 1 -- start start 2024-05-08T04:08:22.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.860+0000 7f3e03e6f640 1 Processor -- start 2024-05-08T04:08:22.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.860+0000 7f4accf5a640 1 -- 172.21.15.170:0/2906882768 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ac002eba0 con 0x7f4ac8070de0 2024-05-08T04:08:22.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.860+0000 7f4accf5a640 1 -- 172.21.15.170:0/2906882768 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0009540 con 0x7f4ac8070de0 2024-05-08T04:08:22.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.861+0000 7f66cd4ca640 1 --2- 172.21.15.170:0/338644855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66c8073990 0x7f66c8071100 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00a940 tx=0x7f66bc02cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f787423f440374e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.861+0000 7f62bdd8b640 1 Processor -- start 2024-05-08T04:08:22.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.861+0000 7f66c77fe640 1 -- 172.21.15.170:0/338644855 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc03a070 con 0x7f66c8073990 2024-05-08T04:08:22.867 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.861+0000 7f66c77fe640 1 -- 172.21.15.170:0/338644855 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66bc02db40 con 0x7f66c8073990 2024-05-08T04:08:22.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.861+0000 7feeb36ed640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7feeac070500 0x7feeac14bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.861+0000 7feeb36ed640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feeac14c000 con 0x7feeac070120 2024-05-08T04:08:22.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.861+0000 7feeb36ed640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feeac072370 con 0x7feeac1454c0 2024-05-08T04:08:22.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.861+0000 7feeb36ed640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7feeac070900 con 0x7feeac070500 2024-05-08T04:08:22.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.861+0000 7f2689d54640 1 Processor -- start 2024-05-08T04:08:22.868 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.861+0000 7f2689d54640 1 -- start start 2024-05-08T04:08:22.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.861+0000 7f66c77fe640 1 -- 172.21.15.170:0/338644855 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc00b040 con 0x7f66c8073990 2024-05-08T04:08:22.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.861+0000 7f4acef5e640 1 -- 172.21.15.170:0/2906882768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ac8070de0 msgr2=0x7f4ac813f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.861+0000 7f4acef5e640 1 --2- 172.21.15.170:0/2906882768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ac8070de0 0x7f4ac813f9c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8074430 tx=0x7f4ac002e060 comp rx=0 tx=0).stop 2024-05-08T04:08:22.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7feeb1489640 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7feeac070120 0x7feeac144db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:55926/0 (socket says 172.21.15.170:55926) 2024-05-08T04:08:22.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7feeb1c8a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7feeac070500 0x7feeac14bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.869 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7feeb0c88640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7feeac1454c0 0x7feeac14b3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.170:50292/0 (socket says 172.21.15.170:50292) 2024-05-08T04:08:22.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7feeb0c88640 1 -- 172.21.15.170:0/4047503698 learned_addr learned my addr 172.21.15.170:0/4047503698 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-05-08T04:08:22.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.861+0000 7f3e03e6f640 1 -- start start 2024-05-08T04:08:22.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7f3e03e6f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3dfc0aa980 0x7f3dfc1435f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7f3e03e6f640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3dfc143b30 0x7f3dfc14dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7feeb1c8a640 1 -- 172.21.15.170:0/4047503698 >> v1:172.21.15.187:6789/0 conn(0x7feeac1454c0 legacy=0x7feeac14b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7feeb1c8a640 1 -- 172.21.15.170:0/4047503698 >> v1:172.21.15.170:6789/0 conn(0x7feeac070120 legacy=0x7feeac144db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7f3e01c0b640 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f3dfc0b6270 0x7f3dfc140a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:55930/0 (socket says 172.21.15.170:55930) 2024-05-08T04:08:22.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7f3e0240c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3dfc143b30 0x7f3dfc14dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7feeb1c8a640 1 -- 172.21.15.170:0/4047503698 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeac072650 con 0x7feeac070500 2024-05-08T04:08:22.871 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.861+0000 7f62bdd8b640 1 -- start start 2024-05-08T04:08:22.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7f62bdd8b640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f62b80afaa0 0x7f62b80712b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7f62bdd8b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62b8140320 0x7f62b80738b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7f2689d54640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f26841477e0 0x7f2684070d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7f2689d54640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f26840712c0 0x7f2684152510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7f2689d54640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26840afab0 con 0x7f2684068c80 2024-05-08T04:08:22.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7f2689d54640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f2684069060 con 0x7f26841477e0 2024-05-08T04:08:22.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7f2689d54640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f268414b1e0 con 0x7f26840712c0 2024-05-08T04:08:22.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7f3e0240c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3dfc143b30 0x7f3dfc14dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:54252/0 (socket says 172.21.15.170:54252) 2024-05-08T04:08:22.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7f3e0140a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3dfc0aa980 0x7f3dfc1435f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7f66cef2d640 1 -- 172.21.15.170:0/338644855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66c8073990 msgr2=0x7f66c8071100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7f66cef2d640 1 --2- 172.21.15.170:0/338644855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66c8073990 0x7f66c8071100 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00a940 tx=0x7f66bc02cd90 comp rx=0 tx=0).stop 2024-05-08T04:08:22.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7fdc96ffd640 1 --2- 172.21.15.170:0/140552821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc80003a30 0x7fdc80003e30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdc980b26e0 tx=0x7fdc80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3bd8a5672c21b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7f62bcd89640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f62b80afaa0 0x7f62b80712b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7f62bcd89640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f62b80afaa0 0x7f62b80712b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49036/0 (socket says 172.21.15.170:49036) 2024-05-08T04:08:22.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7f3e03e6f640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dfc0aadc0 con 0x7f3dfc0b6270 2024-05-08T04:08:22.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7f3e03e6f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3dfc0b6650 con 0x7f3dfc0aa980 2024-05-08T04:08:22.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7f3e01c0b640 1 -- 172.21.15.170:0/251358506 learned_addr learned my addr 172.21.15.170:0/251358506 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-05-08T04:08:22.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7f66cef2d640 1 -- 172.21.15.170:0/338644855 shutdown_connections 2024-05-08T04:08:22.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7f66cef2d640 1 --2- 172.21.15.170:0/338644855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66c8073990 0x7f66c8071100 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7f66cef2d640 1 -- 172.21.15.170:0/338644855 >> 172.21.15.170:0/338644855 conn(0x7f66c80b4e30 msgr2=0x7f66c80b5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:22.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7f62bd58a640 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f62b8140780 0x7f62b8073df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:55944/0 (socket says 172.21.15.170:55944) 2024-05-08T04:08:22.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7f62bcd89640 1 -- 172.21.15.170:0/2463136980 learned_addr learned my addr 172.21.15.170:0/2463136980 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:22.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7f3e03e6f640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f3dfc0ac6b0 con 0x7f3dfc143b30 2024-05-08T04:08:22.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7f62bdd8b640 1 -- 172.21.15.170:0/2463136980 --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62b8072320 con 0x7f62b8140780 2024-05-08T04:08:22.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7f3e00c09640 1 -- 172.21.15.170:0/251358506 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1833747002 0 0) 0x7f3dfc0aadc0 con 0x7f3dfc0b6270 2024-05-08T04:08:22.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7f62bdd8b640 1 -- 172.21.15.170:0/2463136980 --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7f62b813df70 con 0x7f62b80afaa0 2024-05-08T04:08:22.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7f62bdd8b640 1 -- 172.21.15.170:0/2463136980 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f62b8073490 con 0x7f62b8140320 2024-05-08T04:08:22.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7f66cef2d640 1 -- 172.21.15.170:0/338644855 shutdown_connections 2024-05-08T04:08:22.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7f66cef2d640 1 -- 172.21.15.170:0/338644855 wait complete. 2024-05-08T04:08:22.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7f3e00c09640 1 -- 172.21.15.170:0/251358506 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3de8003520 con 0x7f3dfc0b6270 2024-05-08T04:08:22.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7f3e00c09640 1 -- 172.21.15.170:0/251358506 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 604097562 0 0) 0x7f3de8003520 con 0x7f3dfc0b6270 2024-05-08T04:08:22.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7f3e00c09640 1 -- 172.21.15.170:0/251358506 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3dfc143b30 msgr2=0x7f3dfc14dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:22.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7f3e00c09640 1 --2- 172.21.15.170:0/251358506 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3dfc143b30 0x7f3dfc14dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7f3e00c09640 1 -- 172.21.15.170:0/251358506 >> v2:172.21.15.187:3300/0 conn(0x7f3dfc0aa980 msgr2=0x7f3dfc1435f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:22.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7f3e00c09640 1 --2- 172.21.15.170:0/251358506 >> v2:172.21.15.187:3300/0 conn(0x7f3dfc0aa980 0x7f3dfc1435f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7f3e00c09640 1 -- 172.21.15.170:0/251358506 --> v1:172.21.15.170:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dfc0b5890 con 0x7f3dfc0b6270 2024-05-08T04:08:22.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7f3e00c09640 1 -- 172.21.15.170:0/251358506 <== mon.0 v1:172.21.15.170:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1210551688 0 0) 0x7f3df80016f0 con 0x7f3dfc0b6270 2024-05-08T04:08:22.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7f3e00c09640 1 -- 172.21.15.170:0/251358506 >> v1:172.21.15.170:6789/0 conn(0x7f3dfc0b6270 legacy=0x7f3dfc140a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7f3e00c09640 1 --2- 172.21.15.170:0/251358506 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3dfc143b30 0x7f3de8003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7fdc95ffb640 1 -- 172.21.15.170:0/140552821 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c037070 con 0x7fdc80003a30 2024-05-08T04:08:22.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7fdc95ffb640 1 -- 172.21.15.170:0/140552821 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc8c032600 con 0x7fdc80003a30 2024-05-08T04:08:22.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7fdc95ffb640 1 -- 172.21.15.170:0/140552821 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c009370 con 0x7fdc80003a30 2024-05-08T04:08:22.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7f62b7fff640 1 --2- 172.21.15.170:0/2463136980 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62b8140320 0x7f62b80738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.864+0000 7f62b77fe640 1 -- 172.21.15.170:0/2463136980 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1005141560 0 0) 0x7f62b8072320 con 0x7f62b8140780 2024-05-08T04:08:22.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.864+0000 7f62b77fe640 1 -- 172.21.15.170:0/2463136980 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f62a0003520 con 0x7f62b8140780 2024-05-08T04:08:22.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7f2682ffd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f26841477e0 0x7f2684070d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7f26837fe640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f2684068c80 0x7f26841470d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.170:50298/0 (socket says 172.21.15.170:50298) 2024-05-08T04:08:22.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7f26837fe640 1 -- 172.21.15.170:0/4275564350 learned_addr learned my addr 172.21.15.170:0/4275564350 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-05-08T04:08:22.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.862+0000 7f2683fff640 1 --2- 172.21.15.170:0/4275564350 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f26840712c0 0x7f2684152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7f2682ffd640 1 -- 172.21.15.170:0/4275564350 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f26840712c0 msgr2=0x7f2684152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:22.879 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7f2682ffd640 1 --2- 172.21.15.170:0/4275564350 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f26840712c0 0x7f2684152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7f2682ffd640 1 -- 172.21.15.170:0/4275564350 >> v1:172.21.15.187:6789/0 conn(0x7f2684068c80 legacy=0x7f26841470d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7f2682ffd640 1 -- 172.21.15.170:0/4275564350 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2684070730 con 0x7f26841477e0 2024-05-08T04:08:22.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7f4acef5e640 1 -- 172.21.15.170:0/2906882768 shutdown_connections 2024-05-08T04:08:22.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7f4acef5e640 1 --2- 172.21.15.170:0/2906882768 >> v2:172.21.15.170:3300/0 conn(0x7f4ac813ff00 0x7f4ac8140390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7f4acef5e640 1 --2- 172.21.15.170:0/2906882768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ac8070de0 0x7f4ac813f9c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7f4acef5e640 1 -- 172.21.15.170:0/2906882768 >> 172.21.15.170:0/2906882768 conn(0x7f4ac80b4bb0 msgr2=0x7f4ac80b4f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:22.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.864+0000 7f4acef5e640 1 -- 172.21.15.170:0/2906882768 shutdown_connections 2024-05-08T04:08:22.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.864+0000 7f4acef5e640 1 -- 172.21.15.170:0/2906882768 wait complete. 2024-05-08T04:08:22.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7f3e00c09640 1 --2- 172.21.15.170:0/251358506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3de8003f80 0x7f3de8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7f3e00c09640 1 --2- 172.21.15.170:0/251358506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3de8004d40 0x7f3de80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7f3e00c09640 1 -- 172.21.15.170:0/251358506 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc0b6650 con 0x7f3de8003f80 2024-05-08T04:08:22.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7f3e00c09640 1 -- 172.21.15.170:0/251358506 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc0ac6b0 con 0x7f3de8004d40 2024-05-08T04:08:22.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7f3e00c09640 1 -- 172.21.15.170:0/251358506 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f3de800baf0 con 0x7f3dfc143b30 2024-05-08T04:08:22.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.863+0000 7f47693ce640 1 Processor -- start 2024-05-08T04:08:22.881 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.864+0000 7f3e0140a640 1 --2- 172.21.15.170:0/251358506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3de8003f80 0x7f3de8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.864+0000 7feeb1c8a640 1 --2- 172.21.15.170:0/4047503698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feeac070500 0x7feeac14bac0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feea4009d90 tx=0x7feea402bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=379709125b4dd32b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.864+0000 7f47693ce640 1 -- start start 2024-05-08T04:08:22.882 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.864+0000 7f3e01c0b640 1 --2- 172.21.15.170:0/251358506 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3dfc143b30 0x7f3de8003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7fdc97fff640 1 -- 172.21.15.170:0/140552821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc80003a30 msgr2=0x7fdc80003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7fdc97fff640 1 --2- 172.21.15.170:0/140552821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc80003a30 0x7fdc80003e30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdc980b26e0 tx=0x7fdc80005830 comp rx=0 tx=0).stop 2024-05-08T04:08:22.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7fdc977fe640 1 -- 172.21.15.170:0/140552821 reap_dead start 2024-05-08T04:08:22.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.864+0000 7feea3fff640 1 -- 172.21.15.170:0/4047503698 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea4039070 con 0x7feeac070500 2024-05-08T04:08:22.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.864+0000 7feea3fff640 1 -- 172.21.15.170:0/4047503698 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feea402cab0 con 0x7feeac070500 2024-05-08T04:08:22.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7feeb36ed640 1 -- 172.21.15.170:0/4047503698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feeac070500 msgr2=0x7feeac14bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.883 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7feeb36ed640 1 --2- 172.21.15.170:0/4047503698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feeac070500 0x7feeac14bac0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feea4009d90 tx=0x7feea402bd80 comp rx=0 tx=0).stop 2024-05-08T04:08:22.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.864+0000 7f62bcd89640 1 -- 172.21.15.170:0/2463136980 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62b8140320 msgr2=0x7f62b80738b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:22.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.864+0000 7f62bcd89640 1 --2- 172.21.15.170:0/2463136980 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62b8140320 0x7f62b80738b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.864+0000 7f62bcd89640 1 -- 172.21.15.170:0/2463136980 >> v1:172.21.15.170:6789/0 conn(0x7f62b8140780 legacy=0x7f62b8073df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.864+0000 7f62bcd89640 1 -- 172.21.15.170:0/2463136980 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b8071d90 con 0x7f62b80afaa0 2024-05-08T04:08:22.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.864+0000 7f62b7fff640 1 --2- 172.21.15.170:0/2463136980 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62b8140320 0x7f62b80738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:08:22.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7f4acef5e640 1 Processor -- start 2024-05-08T04:08:22.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7f4acef5e640 1 -- start start 2024-05-08T04:08:22.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.864+0000 7f3e0240c640 1 --2- 172.21.15.170:0/251358506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3de8004d40 0x7f3de80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.864+0000 7f3e0140a640 1 -- 172.21.15.170:0/251358506 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3dfc143b30 msgr2=0x7f3de8003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:22.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.864+0000 7f3e0140a640 1 --2- 172.21.15.170:0/251358506 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3dfc143b30 0x7f3de8003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.864+0000 7f3e0140a640 1 -- 172.21.15.170:0/251358506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3de8004d40 msgr2=0x7f3de80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:22.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.864+0000 7f3e0140a640 1 --2- 172.21.15.170:0/251358506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3de8004d40 0x7f3de80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.864+0000 7f3e0140a640 1 -- 172.21.15.170:0/251358506 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dfc0b5890 con 0x7f3de8003f80 2024-05-08T04:08:22.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7f3e0240c640 1 --2- 172.21.15.170:0/251358506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3de8004d40 0x7f3de80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:08:22.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7f47693ce640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f47640700a0 0x7f4764144b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7f47693ce640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4764145040 0x7f4764147b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7f47693ce640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4764148070 0x7f476414b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7f47693ce640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7f47640710b0 con 0x7f47640700a0 2024-05-08T04:08:22.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7f47693ce640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f47640725d0 con 0x7f4764148070 2024-05-08T04:08:22.885 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7f47693ce640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f47641454a0 con 0x7f4764145040 2024-05-08T04:08:22.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7f4762ffd640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f47640700a0 0x7f4764144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7f4762ffd640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f47640700a0 0x7f4764144b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49048/0 (socket says 172.21.15.170:49048) 2024-05-08T04:08:22.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7f4762ffd640 1 -- 172.21.15.170:0/1749360084 learned_addr learned my addr 172.21.15.170:0/1749360084 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:22.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7f47637fe640 1 --2- 172.21.15.170:0/1749360084 >> v2:172.21.15.187:3300/0 conn(0x7f4764148070 0x7f476414b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7f475a7fc640 1 --2- 172.21.15.170:0/1749360084 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4764145040 0x7f4764147b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7f4762ffd640 1 -- 172.21.15.170:0/1749360084 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4764145040 msgr2=0x7f4764147b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:22.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7f4762ffd640 1 --2- 172.21.15.170:0/1749360084 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4764145040 0x7f4764147b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7f4762ffd640 1 -- 172.21.15.170:0/1749360084 >> v2:172.21.15.187:3300/0 conn(0x7f4764148070 msgr2=0x7f476414b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:22.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7f4762ffd640 1 --2- 172.21.15.170:0/1749360084 >> v2:172.21.15.187:3300/0 conn(0x7f4764148070 0x7f476414b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7f4762ffd640 1 -- 172.21.15.170:0/1749360084 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47640743b0 con 0x7f47640700a0 2024-05-08T04:08:22.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.864+0000 7f66cef2d640 1 Processor -- start 2024-05-08T04:08:22.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.864+0000 7f66cef2d640 1 -- start start 2024-05-08T04:08:22.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7f66cef2d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66c8070320 0x7f66c80ae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7f66cef2d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f66c8073990 0x7f66c8139e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7f3e0140a640 1 --2- 172.21.15.170:0/251358506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3de8003f80 0x7f3de8004800 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3dec009ac0 tx=0x7f3de8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c401485b508486e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7f66cef2d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66c813a380 0x7f66c81d4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7f66cef2d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f66c80b5950 con 0x7f66c813a380 2024-05-08T04:08:22.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7f66cef2d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f66c80716f0 con 0x7f66c8070320 2024-05-08T04:08:22.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7f66cef2d640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f66c80b5ad0 con 0x7f66c8073990 2024-05-08T04:08:22.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7feeb36ed640 1 -- 172.21.15.170:0/4047503698 shutdown_connections 2024-05-08T04:08:22.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7feeb36ed640 1 --2- 172.21.15.170:0/4047503698 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feeac070500 0x7feeac14bac0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7feeb36ed640 1 -- 172.21.15.170:0/4047503698 >> 172.21.15.170:0/4047503698 conn(0x7feeac0b4e30 msgr2=0x7feeac0b5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:22.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.865+0000 7f2682ffd640 1 --2- 172.21.15.170:0/4275564350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26841477e0 0x7f2684070d80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f26780097d0 tx=0x7f267802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1f823e0ec1945c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.866+0000 7f66cccc9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66c8070320 0x7f66c80ae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.866+0000 7fdc97fff640 1 -- 172.21.15.170:0/140552821 shutdown_connections 2024-05-08T04:08:22.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.866+0000 7fdc97fff640 1 -- 172.21.15.170:0/140552821 >> 172.21.15.170:0/140552821 conn(0x7fdc980b4bb0 msgr2=0x7fdc980b4f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:22.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.866+0000 7f66cccc9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66c8070320 0x7f66c80ae4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.170:47068/0 (socket says 172.21.15.170:47068) 2024-05-08T04:08:22.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.866+0000 7f62bcd89640 1 --2- 172.21.15.170:0/2463136980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62b80afaa0 0x7f62b80712b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00dc90 tx=0x7f62ac02fd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ed10e732321c0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.866+0000 7f26827fc640 1 -- 172.21.15.170:0/4275564350 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2678031780 con 0x7f26841477e0 2024-05-08T04:08:22.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.866+0000 7f26827fc640 1 -- 172.21.15.170:0/4275564350 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2678037070 con 0x7f26841477e0 2024-05-08T04:08:22.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.866+0000 7f66cccc9640 1 -- 172.21.15.170:0/2611151330 learned_addr learned my addr 172.21.15.170:0/2611151330 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:22.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.866+0000 7f66c7fff640 1 --2- 172.21.15.170:0/2611151330 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f66c8073990 0x7f66c8139e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.866+0000 7f3e01c0b640 1 --2- 172.21.15.170:0/251358506 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3dfc143b30 0x7f3de8003a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:08:22.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.866+0000 7f4acef5e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4ac80700a0 0x7f4ac81c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.866+0000 7f4acef5e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ac8070de0 0x7f4ac80add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.866+0000 7f66cd4ca640 1 --2- 172.21.15.170:0/2611151330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66c813a380 0x7f66c81d4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.866+0000 7f4acef5e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ac813ff00 0x7f4ac813a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.866+0000 7f4acef5e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac8070480 con 0x7f4ac8070de0 2024-05-08T04:08:22.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.866+0000 7f4acef5e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac8071250 con 0x7f4ac813ff00 2024-05-08T04:08:22.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.866+0000 7f4acef5e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f4ac8070600 con 0x7f4ac80700a0 2024-05-08T04:08:22.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.866+0000 7feeb36ed640 1 -- 172.21.15.170:0/4047503698 shutdown_connections 2024-05-08T04:08:22.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.866+0000 7feeb36ed640 1 -- 172.21.15.170:0/4047503698 wait complete. 2024-05-08T04:08:22.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.866+0000 7f26827fc640 1 -- 172.21.15.170:0/4275564350 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2678031b40 con 0x7f26841477e0 2024-05-08T04:08:22.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.866+0000 7f3e00c09640 1 -- 172.21.15.170:0/251358506 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dec039070 con 0x7f3de8003f80 2024-05-08T04:08:22.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.866+0000 7f3e00c09640 1 -- 172.21.15.170:0/251358506 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dec02c710 con 0x7f3de8003f80 2024-05-08T04:08:22.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.866+0000 7f3e00c09640 1 -- 172.21.15.170:0/251358506 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dec009690 con 0x7f3de8003f80 2024-05-08T04:08:22.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.866+0000 7f66cccc9640 1 -- 172.21.15.170:0/2611151330 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f66c8073990 msgr2=0x7f66c8139e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.866+0000 7f66cccc9640 1 --2- 172.21.15.170:0/2611151330 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f66c8073990 0x7f66c8139e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.866+0000 7f66cccc9640 1 -- 172.21.15.170:0/2611151330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66c813a380 msgr2=0x7f66c81d4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.866+0000 7f66cccc9640 1 --2- 172.21.15.170:0/2611151330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66c813a380 0x7f66c81d4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.866+0000 7f66cccc9640 1 -- 172.21.15.170:0/2611151330 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c811cc20 con 0x7f66c8070320 2024-05-08T04:08:22.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.866+0000 7f62b77fe640 1 -- 172.21.15.170:0/2463136980 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac0308a0 con 0x7f62b80afaa0 2024-05-08T04:08:22.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.866+0000 7f4acdf5c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4ac80700a0 0x7f4ac81c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.866+0000 7f4acdf5c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4ac80700a0 0x7f4ac81c8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:54308/0 (socket says 172.21.15.170:54308) 2024-05-08T04:08:22.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.866+0000 7f4acdf5c640 1 -- 172.21.15.170:0/4218083282 learned_addr learned my addr 172.21.15.170:0/4218083282 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:22.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.867+0000 7f62b77fe640 1 -- 172.21.15.170:0/2463136980 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62ac030a40 con 0x7f62b80afaa0 2024-05-08T04:08:22.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.867+0000 7f62b77fe640 1 -- 172.21.15.170:0/2463136980 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac030df0 con 0x7f62b80afaa0 2024-05-08T04:08:22.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.867+0000 7f4acdf5c640 1 -- 172.21.15.170:0/4218083282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ac813ff00 msgr2=0x7f4ac813a400 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T04:08:22.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.867+0000 7f4acdf5c640 1 --2- 172.21.15.170:0/4218083282 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ac813ff00 0x7f4ac813a400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.867+0000 7f4acd75b640 1 --2- 172.21.15.170:0/4218083282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ac8070de0 0x7f4ac80add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.867+0000 7f4acdf5c640 1 -- 172.21.15.170:0/4218083282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ac8070de0 msgr2=0x7f4ac80add10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.867+0000 7f4acdf5c640 1 --2- 172.21.15.170:0/4218083282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ac8070de0 0x7f4ac80add10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.866+0000 7f4762ffd640 1 --2- 172.21.15.170:0/1749360084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47640700a0 0x7f4764144b00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4754009870 tx=0x7f475402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd272e8dba6fb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.867+0000 7f47627fc640 1 -- 172.21.15.170:0/1749360084 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47540317c0 con 0x7f47640700a0 2024-05-08T04:08:22.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.867+0000 7f4acdf5c640 1 -- 172.21.15.170:0/4218083282 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac80ad610 con 0x7f4ac80700a0 2024-05-08T04:08:22.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.867+0000 7f47627fc640 1 -- 172.21.15.170:0/1749360084 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4754037070 con 0x7f47640700a0 2024-05-08T04:08:22.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.867+0000 7f4acd75b640 1 --2- 172.21.15.170:0/4218083282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ac8070de0 0x7f4ac80add10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:08:22.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.867+0000 7f3e03e6f640 1 -- 172.21.15.170:0/251358506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3de8003f80 msgr2=0x7f3de8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.867+0000 7f2689d54640 1 -- 172.21.15.170:0/4275564350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26841477e0 msgr2=0x7f2684070d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.867+0000 7f2689d54640 1 --2- 172.21.15.170:0/4275564350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26841477e0 0x7f2684070d80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f26780097d0 tx=0x7f267802b820 comp rx=0 tx=0).stop 2024-05-08T04:08:22.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.867+0000 7f3e03e6f640 1 --2- 172.21.15.170:0/251358506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3de8003f80 0x7f3de8004800 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3dec009ac0 tx=0x7f3de8005830 comp rx=0 tx=0).stop 2024-05-08T04:08:22.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.867+0000 7f3e0240c640 1 -- 172.21.15.170:0/251358506 reap_dead start 2024-05-08T04:08:22.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.867+0000 7f66cccc9640 1 --2- 172.21.15.170:0/2611151330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66c8070320 0x7f66c80ae4a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f66b800c8d0 tx=0x7f66b800cd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.867+0000 7f66c57fa640 1 -- 172.21.15.170:0/2611151330 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b8008040 con 0x7f66c8070320 2024-05-08T04:08:22.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.867+0000 7f66c57fa640 1 -- 172.21.15.170:0/2611151330 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66b8004c00 con 0x7f66c8070320 2024-05-08T04:08:22.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.867+0000 7f66c57fa640 1 -- 172.21.15.170:0/2611151330 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b800ae10 con 0x7f66c8070320 2024-05-08T04:08:22.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.867+0000 7f66cef2d640 1 -- 172.21.15.170:0/2611151330 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66c80ae040 con 0x7f66c8070320 2024-05-08T04:08:22.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.867+0000 7f3e03e6f640 1 -- 172.21.15.170:0/251358506 shutdown_connections 2024-05-08T04:08:22.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.867+0000 7f3e03e6f640 1 -- 172.21.15.170:0/251358506 >> 172.21.15.170:0/251358506 conn(0x7f3dfc076050 msgr2=0x7f3dfc076460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:22.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.867+0000 7f66cef2d640 1 -- 172.21.15.170:0/2611151330 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66c80ae9e0 con 0x7f66c8070320 2024-05-08T04:08:22.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.867+0000 7f47627fc640 1 -- 172.21.15.170:0/1749360084 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4754031b80 con 0x7f47640700a0 2024-05-08T04:08:22.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.867+0000 7fdc97fff640 1 -- 172.21.15.170:0/140552821 shutdown_connections 2024-05-08T04:08:22.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.867+0000 7f3e03e6f640 1 -- 172.21.15.170:0/251358506 shutdown_connections 2024-05-08T04:08:22.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.868+0000 7f3e03e6f640 1 -- 172.21.15.170:0/251358506 wait complete. 2024-05-08T04:08:22.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.867+0000 7f4acdf5c640 1 --2- 172.21.15.170:0/4218083282 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4ac80700a0 0x7f4ac81c8dc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4ac400c760 tx=0x7f4ac400cc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.868+0000 7feeb36ed640 1 Processor -- start 2024-05-08T04:08:22.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.868+0000 7f62bdd8b640 1 -- 172.21.15.170:0/2463136980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62b80afaa0 msgr2=0x7f62b80712b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.868+0000 7f62bdd8b640 1 --2- 172.21.15.170:0/2463136980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62b80afaa0 0x7f62b80712b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00dc90 tx=0x7f62ac02fd00 comp rx=0 tx=0).stop 2024-05-08T04:08:22.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.868+0000 7f62bdd8b640 1 -- 172.21.15.170:0/2463136980 shutdown_connections 2024-05-08T04:08:22.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.868+0000 7f62bdd8b640 1 --2- 172.21.15.170:0/2463136980 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62b8140320 0x7f62b80738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.868+0000 7f62bdd8b640 1 --2- 172.21.15.170:0/2463136980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62b80afaa0 0x7f62b80712b0 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00dc90 tx=0x7f62ac02fd00 comp rx=0 tx=0).stop 2024-05-08T04:08:22.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.868+0000 7f62bdd8b640 1 -- 172.21.15.170:0/2463136980 >> 172.21.15.170:0/2463136980 conn(0x7f62b80b5dd0 msgr2=0x7f62b80b61e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:22.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.868+0000 7f4abe7fc640 1 -- 172.21.15.170:0/4218083282 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac40052d0 con 0x7f4ac80700a0 2024-05-08T04:08:22.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.868+0000 7f47693ce640 1 -- 172.21.15.170:0/1749360084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47640700a0 msgr2=0x7f4764144b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.868+0000 7f47693ce640 1 --2- 172.21.15.170:0/1749360084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47640700a0 0x7f4764144b00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4754009870 tx=0x7f475402b860 comp rx=0 tx=0).stop 2024-05-08T04:08:22.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.868+0000 7f4acef5e640 1 -- 172.21.15.170:0/4218083282 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ac80ad8f0 con 0x7f4ac80700a0 2024-05-08T04:08:22.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.868+0000 7f4acef5e640 1 -- 172.21.15.170:0/4218083282 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ac813ab20 con 0x7f4ac80700a0 2024-05-08T04:08:22.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.868+0000 7f62bdd8b640 1 -- 172.21.15.170:0/2463136980 shutdown_connections 2024-05-08T04:08:22.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.868+0000 7f47693ce640 1 -- 172.21.15.170:0/1749360084 shutdown_connections 2024-05-08T04:08:22.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.868+0000 7f47693ce640 1 --2- 172.21.15.170:0/1749360084 >> v2:172.21.15.187:3300/0 conn(0x7f4764148070 0x7f476414b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.868+0000 7f47693ce640 1 --2- 172.21.15.170:0/1749360084 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4764145040 0x7f4764147b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.868+0000 7f47693ce640 1 --2- 172.21.15.170:0/1749360084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47640700a0 0x7f4764144b00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.868+0000 7f47693ce640 1 -- 172.21.15.170:0/1749360084 >> 172.21.15.170:0/1749360084 conn(0x7f47640b5250 msgr2=0x7f47640b5660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:22.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.868+0000 7fdc97fff640 1 -- 172.21.15.170:0/140552821 wait complete. 2024-05-08T04:08:22.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.868+0000 7feeb36ed640 1 -- start start 2024-05-08T04:08:22.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.869+0000 7feeb36ed640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feeac070120 0x7feeac147c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.869+0000 7feeb36ed640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7feeac070500 0x7feeac0abd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.869+0000 7feeb1489640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feeac070120 0x7feeac147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.869+0000 7feeb1489640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feeac070120 0x7feeac147c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49076/0 (socket says 172.21.15.170:49076) 2024-05-08T04:08:22.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.869+0000 7feeb36ed640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feeac1454c0 0x7feeac0aee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.869+0000 7feeb36ed640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7feeac153470 con 0x7feeac070120 2024-05-08T04:08:22.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.869+0000 7feeb36ed640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7feeac071550 con 0x7feeac1454c0 2024-05-08T04:08:22.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.869+0000 7feeb36ed640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7feeac1535f0 con 0x7feeac070500 2024-05-08T04:08:22.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.869+0000 7feeb1489640 1 -- 172.21.15.170:0/844936287 learned_addr learned my addr 172.21.15.170:0/844936287 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:22.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.869+0000 7feeb1c8a640 1 --2- 172.21.15.170:0/844936287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feeac1454c0 0x7feeac0aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.870+0000 7feeb1489640 1 -- 172.21.15.170:0/844936287 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7feeac070500 msgr2=0x7feeac0abd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.870+0000 7feeb1489640 1 --2- 172.21.15.170:0/844936287 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7feeac070500 0x7feeac0abd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.870+0000 7feeb1489640 1 -- 172.21.15.170:0/844936287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feeac1454c0 msgr2=0x7feeac0aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.870+0000 7feeb1489640 1 --2- 172.21.15.170:0/844936287 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feeac1454c0 0x7feeac0aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.870+0000 7feeb1489640 1 -- 172.21.15.170:0/844936287 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeac0ae090 con 0x7feeac070120 2024-05-08T04:08:22.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.869+0000 7f4abe7fc640 1 -- 172.21.15.170:0/4218083282 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ac4005490 con 0x7f4ac80700a0 2024-05-08T04:08:22.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.869+0000 7f3e03e6f640 1 Processor -- start 2024-05-08T04:08:22.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.870+0000 7f3e03e6f640 1 -- start start 2024-05-08T04:08:22.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.869+0000 7f66cef2d640 1 -- 172.21.15.170:0/2611151330 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f66c806b400 con 0x7f66c8070320 2024-05-08T04:08:22.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.870+0000 7feeb1489640 1 --2- 172.21.15.170:0/844936287 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feeac070120 0x7feeac147c30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00c7b0 tx=0x7fee9c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.869+0000 7fdc97fff640 1 Processor -- start 2024-05-08T04:08:22.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.869+0000 7fdc97fff640 1 -- start start 2024-05-08T04:08:22.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.870+0000 7f4abe7fc640 1 -- 172.21.15.170:0/4218083282 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac400a880 con 0x7f4ac80700a0 2024-05-08T04:08:22.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.869+0000 7f2689d54640 1 -- 172.21.15.170:0/4275564350 shutdown_connections 2024-05-08T04:08:22.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.868+0000 7f62bdd8b640 1 -- 172.21.15.170:0/2463136980 wait complete. 2024-05-08T04:08:22.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.870+0000 7f62bdd8b640 1 Processor -- start 2024-05-08T04:08:22.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.869+0000 7f2689d54640 1 --2- 172.21.15.170:0/4275564350 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f26840712c0 0x7f2684152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.871+0000 7f3e03e6f640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3de8003f80 0x7f3dfc1dc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.871+0000 7f4acef5e640 1 -- 172.21.15.170:0/4218083282 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4ab0000f80 con 0x7f4ac80700a0 2024-05-08T04:08:22.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.870+0000 7fdc97fff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdc80003a30 0x7fdc98146460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.870+0000 7fdc97fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc981e0860 0x7fdc98127660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.870+0000 7fdc97fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc98127ba0 0x7fdc981dd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.870+0000 7fdc97fff640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdc98070310 con 0x7fdc98127ba0 2024-05-08T04:08:22.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.870+0000 7fdc97fff640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdc98070ab0 con 0x7fdc981e0860 2024-05-08T04:08:22.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.870+0000 7fdc97fff640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fdc980af970 con 0x7fdc80003a30 2024-05-08T04:08:22.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.871+0000 7feea1ffb640 1 -- 172.21.15.170:0/844936287 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c018070 con 0x7feeac070120 2024-05-08T04:08:22.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.868+0000 7f47693ce640 1 -- 172.21.15.170:0/1749360084 shutdown_connections 2024-05-08T04:08:22.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.868+0000 7f47693ce640 1 -- 172.21.15.170:0/1749360084 wait complete. 2024-05-08T04:08:22.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.871+0000 7f3e01c0b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3de8003f80 0x7f3dfc1dc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.871+0000 7fdc967fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc981e0860 0x7fdc98127660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.869+0000 7f2689d54640 1 --2- 172.21.15.170:0/4275564350 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f26841477e0 0x7f2684070d80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.869+0000 7f2689d54640 1 -- 172.21.15.170:0/4275564350 >> 172.21.15.170:0/4275564350 conn(0x7f26840b5c20 msgr2=0x7f26840b20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:22.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.871+0000 7f2689d54640 1 -- 172.21.15.170:0/4275564350 shutdown_connections 2024-05-08T04:08:22.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.871+0000 7fdc967fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc981e0860 0x7fdc98127660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.170:47090/0 (socket says 172.21.15.170:47090) 2024-05-08T04:08:22.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.871+0000 7fdc967fc640 1 -- 172.21.15.170:0/2187537070 learned_addr learned my addr 172.21.15.170:0/2187537070 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:22.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.871+0000 7f3e01c0b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3de8003f80 0x7f3dfc1dc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:54328/0 (socket says 172.21.15.170:54328) 2024-05-08T04:08:22.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.871+0000 7f3e03e6f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3dfc1dc6b0 0x7f3dfc0b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.871+0000 7feeb36ed640 1 -- 172.21.15.170:0/844936287 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeac0ae370 con 0x7feeac070120 2024-05-08T04:08:22.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.871+0000 7feeb36ed640 1 -- 172.21.15.170:0/844936287 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeac0aa690 con 0x7feeac070120 2024-05-08T04:08:22.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.871+0000 7feea1ffb640 1 -- 172.21.15.170:0/844936287 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee9c0072d0 con 0x7feeac070120 2024-05-08T04:08:22.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.871+0000 7f3e01c0b640 1 -- 172.21.15.170:0/1966442353 learned_addr learned my addr 172.21.15.170:0/1966442353 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:22.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.871+0000 7f3e03e6f640 1 --2- 172.21.15.170:0/1966442353 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3dfc0b10d0 0x7f3dfc1d8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.872+0000 7f3e03e6f640 1 -- 172.21.15.170:0/1966442353 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc143f80 con 0x7f3dfc1dc6b0 2024-05-08T04:08:22.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.872+0000 7f3e0240c640 1 --2- 172.21.15.170:0/1966442353 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3dfc0b10d0 0x7f3dfc1d8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.872+0000 7f3e03e6f640 1 -- 172.21.15.170:0/1966442353 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc0ab570 con 0x7f3dfc0b10d0 2024-05-08T04:08:22.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.870+0000 7f62bdd8b640 1 -- start start 2024-05-08T04:08:22.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.872+0000 7f62bdd8b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62b8140320 0x7f62b81e0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.872+0000 7f62bdd8b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62b8140780 0x7f62b80b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.872+0000 7f62bdd8b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62b8136a10 0x7f62b80b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.872+0000 7f62bdd8b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f62b8075db0 con 0x7f62b8136a10 2024-05-08T04:08:22.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.872+0000 7f62bdd8b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f62b8075c30 con 0x7f62b8140780 2024-05-08T04:08:22.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.872+0000 7f62bdd8b640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f62b8075f30 con 0x7f62b8140320 2024-05-08T04:08:22.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.872+0000 7f3e03e6f640 1 -- 172.21.15.170:0/1966442353 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f3dfc14beb0 con 0x7f3de8003f80 2024-05-08T04:08:22.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.872+0000 7feea1ffb640 1 -- 172.21.15.170:0/844936287 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c0047e0 con 0x7feeac070120 2024-05-08T04:08:22.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.872+0000 7fee977fe640 1 -- 172.21.15.170:0/844936287 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feeac0aea10 con 0x7feeac070120 2024-05-08T04:08:22.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.872+0000 7f62bcd89640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62b8140320 0x7f62b81e0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.872+0000 7f62b7fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62b8140780 0x7f62b80b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.872+0000 7f62b7fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62b8140780 0x7f62b80b3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.170:47126/0 (socket says 172.21.15.170:47126) 2024-05-08T04:08:22.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.872+0000 7f66c57fa640 1 -- 172.21.15.170:0/2611151330 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f66b8007750 con 0x7f66c8070320 2024-05-08T04:08:22.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.871+0000 7f4abe7fc640 1 -- 172.21.15.170:0/4218083282 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f4ac400aaa0 con 0x7f4ac80700a0 2024-05-08T04:08:22.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.872+0000 7f62b7fff640 1 -- 172.21.15.170:0/154066009 learned_addr learned my addr 172.21.15.170:0/154066009 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:22.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.872+0000 7f4abe7fc640 1 --2- 172.21.15.170:0/4218083282 >> v2:172.21.15.187:6816/101007897 conn(0x7f4aac076360 0x7f4aac078820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.873+0000 7f3e0240c640 1 -- 172.21.15.170:0/1966442353 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3de8003f80 msgr2=0x7f3dfc1dc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.872+0000 7f4abe7fc640 1 -- 172.21.15.170:0/4218083282 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 4090+0+0 (secure 0 0 0) 0x7f4ac4096c30 con 0x7f4ac80700a0 2024-05-08T04:08:22.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.872+0000 7f4abe7fc640 1 --2- 172.21.15.170:0/4218083282 >> v2:172.21.15.187:6808/1425849822 conn(0x7f4aac07c340 0x7f4aac07e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.872+0000 7f4abe7fc640 1 -- 172.21.15.170:0/4218083282 --> v2:172.21.15.187:6808/1425849822 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4aac07edd0 con 0x7f4aac07c340 2024-05-08T04:08:22.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.873+0000 7f3e0240c640 1 --2- 172.21.15.170:0/1966442353 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3de8003f80 0x7f3dfc1dc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.873+0000 7f3e0240c640 1 -- 172.21.15.170:0/1966442353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3dfc1dc6b0 msgr2=0x7f3dfc0b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.873+0000 7f3e0240c640 1 --2- 172.21.15.170:0/1966442353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3dfc1dc6b0 0x7f3dfc0b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.873+0000 7f3e0240c640 1 -- 172.21.15.170:0/1966442353 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dfc0b2630 con 0x7f3dfc0b10d0 2024-05-08T04:08:22.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.873+0000 7f3e01c0b640 1 --2- 172.21.15.170:0/1966442353 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3de8003f80 0x7f3dfc1dc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:08:22.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.873+0000 7f62b7fff640 1 -- 172.21.15.170:0/154066009 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62b8140320 msgr2=0x7f62b81e0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.870+0000 7f47693ce640 1 Processor -- start 2024-05-08T04:08:22.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.870+0000 7f47693ce640 1 -- start start 2024-05-08T04:08:22.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.873+0000 7f62b7fff640 1 --2- 172.21.15.170:0/154066009 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f62b8140320 0x7f62b81e0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.873+0000 7f62b7fff640 1 -- 172.21.15.170:0/154066009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62b8136a10 msgr2=0x7f62b80b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.873+0000 7f62b7fff640 1 --2- 172.21.15.170:0/154066009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62b8136a10 0x7f62b80b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.873+0000 7f62b7fff640 1 -- 172.21.15.170:0/154066009 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b81e0a20 con 0x7f62b8140780 2024-05-08T04:08:22.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.871+0000 7fdc96ffd640 1 --2- 172.21.15.170:0/2187537070 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdc80003a30 0x7fdc98146460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.871+0000 7fdc96ffd640 1 -- 172.21.15.170:0/2187537070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc981e0860 msgr2=0x7fdc98127660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.871+0000 7fdc96ffd640 1 --2- 172.21.15.170:0/2187537070 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdc981e0860 0x7fdc98127660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.871+0000 7fdc96ffd640 1 -- 172.21.15.170:0/2187537070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc98127ba0 msgr2=0x7fdc981dd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.871+0000 7fdc96ffd640 1 --2- 172.21.15.170:0/2187537070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc98127ba0 0x7fdc981dd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.871+0000 7fdc96ffd640 1 -- 172.21.15.170:0/2187537070 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc98129d30 con 0x7fdc80003a30 2024-05-08T04:08:22.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.872+0000 7fdc96ffd640 1 --2- 172.21.15.170:0/2187537070 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdc80003a30 0x7fdc98146460 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdc8000c110 tx=0x7fdc8c0046b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.873+0000 7fdc7f7fe640 1 -- 172.21.15.170:0/2187537070 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c041030 con 0x7fdc80003a30 2024-05-08T04:08:22.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.873+0000 7fdc7f7fe640 1 -- 172.21.15.170:0/2187537070 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc8c032400 con 0x7fdc80003a30 2024-05-08T04:08:22.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.873+0000 7f62b7fff640 1 --2- 172.21.15.170:0/154066009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62b8140780 0x7f62b80b3d80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f62a800c7f0 tx=0x7f62a800cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.873+0000 7f4ace75d640 1 --2- 172.21.15.170:0/4218083282 >> v2:172.21.15.187:6808/1425849822 conn(0x7f4aac07c340 0x7f4aac07e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.873+0000 7f3e0240c640 1 --2- 172.21.15.170:0/1966442353 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3dfc0b10d0 0x7f3dfc1d8670 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc0745c0 tx=0x7f3df400f4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.873+0000 7f66c57fa640 1 --2- 172.21.15.170:0/2611151330 >> v2:172.21.15.187:6816/101007897 conn(0x7f66b407a7a0 0x7f66b407cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.871+0000 7f2689d54640 1 -- 172.21.15.170:0/4275564350 wait complete. 2024-05-08T04:08:22.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.873+0000 7f2689d54640 1 Processor -- start 2024-05-08T04:08:22.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.873+0000 7f2689d54640 1 -- start start 2024-05-08T04:08:22.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.871+0000 7f47693ce640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f47640700a0 0x7f4764147540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.871+0000 7f4762ffd640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f47640700a0 0x7f4764147540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.872+0000 7f47693ce640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4764145040 0x7f47641d0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.872+0000 7f475a7fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4764145040 0x7f47641d0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.872+0000 7f475a7fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4764145040 0x7f47641d0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.170:47112/0 (socket says 172.21.15.170:47112) 2024-05-08T04:08:22.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.873+0000 7f4762ffd640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f47640700a0 0x7f4764147540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:54342/0 (socket says 172.21.15.170:54342) 2024-05-08T04:08:22.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.873+0000 7f47693ce640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4764148070 0x7f47641d05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.873+0000 7f47693ce640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4764148680 con 0x7f4764148070 2024-05-08T04:08:22.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.873+0000 7f47693ce640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4764148500 con 0x7f4764145040 2024-05-08T04:08:22.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.873+0000 7f47693ce640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f4764070480 con 0x7f47640700a0 2024-05-08T04:08:22.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.873+0000 7f62b57fa640 1 -- 172.21.15.170:0/154066009 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a8015070 con 0x7f62b8140780 2024-05-08T04:08:22.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.873+0000 7f475a7fc640 1 -- 172.21.15.170:0/2999809509 learned_addr learned my addr 172.21.15.170:0/2999809509 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:22.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.873+0000 7f4acd75b640 1 --2- 172.21.15.170:0/4218083282 >> v2:172.21.15.187:6816/101007897 conn(0x7f4aac076360 0x7f4aac078820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.873+0000 7f62b57fa640 1 -- 172.21.15.170:0/154066009 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62a8007a30 con 0x7f62b8140780 2024-05-08T04:08:22.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.873+0000 7f66c7fff640 1 --2- 172.21.15.170:0/2611151330 >> v2:172.21.15.187:6816/101007897 conn(0x7f66b407a7a0 0x7f66b407cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.873+0000 7f62bdd8b640 1 -- 172.21.15.170:0/154066009 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b81e0f40 con 0x7f62b8140780 2024-05-08T04:08:22.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.874+0000 7f62bdd8b640 1 -- 172.21.15.170:0/154066009 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b80aead0 con 0x7f62b8140780 2024-05-08T04:08:22.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.874+0000 7f4abe7fc640 1 -- 172.21.15.170:0/4218083282 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f4ac405c410 con 0x7f4ac80700a0 2024-05-08T04:08:22.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.873+0000 7f3df27fc640 1 -- 172.21.15.170:0/1966442353 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df401c070 con 0x7f3dfc0b10d0 2024-05-08T04:08:22.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.874+0000 7f3e03e6f640 1 -- 172.21.15.170:0/1966442353 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dfc0ad880 con 0x7f3dfc0b10d0 2024-05-08T04:08:22.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.874+0000 7f47637fe640 1 --2- 172.21.15.170:0/2999809509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4764148070 0x7f47641d05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.874+0000 7f3e03e6f640 1 -- 172.21.15.170:0/1966442353 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dfc0b1730 con 0x7f3dfc0b10d0 2024-05-08T04:08:22.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.874+0000 7f3df27fc640 1 -- 172.21.15.170:0/1966442353 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3df4017400 con 0x7f3dfc0b10d0 2024-05-08T04:08:22.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.874+0000 7f47637fe640 1 -- 172.21.15.170:0/2999809509 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f47640700a0 msgr2=0x7f4764147540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.874+0000 7f47637fe640 1 --2- 172.21.15.170:0/2999809509 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f47640700a0 0x7f4764147540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.874+0000 7f47637fe640 1 -- 172.21.15.170:0/2999809509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4764145040 msgr2=0x7f47641d0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.874+0000 7f47637fe640 1 --2- 172.21.15.170:0/2999809509 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4764145040 0x7f47641d0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.874+0000 7f47637fe640 1 -- 172.21.15.170:0/2999809509 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47641cf6f0 con 0x7f4764148070 2024-05-08T04:08:22.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.874+0000 7f66c57fa640 1 -- 172.21.15.170:0/2611151330 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4090+0+0 (secure 0 0 0) 0x7f66b8096b60 con 0x7f66c8070320 2024-05-08T04:08:22.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.874+0000 7f66c57fa640 1 --2- 172.21.15.170:0/2611151330 >> v2:172.21.15.187:6804/2815417961 conn(0x7f66b4080780 0x7f66b4082ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.874+0000 7f66c57fa640 1 -- 172.21.15.170:0/2611151330 --> v2:172.21.15.187:6804/2815417961 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f66b8015e50 con 0x7f66b4080780 2024-05-08T04:08:22.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.874+0000 7f66c57fa640 1 -- 172.21.15.170:0/2611151330 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f66b809b2a0 con 0x7f66c8070320 2024-05-08T04:08:22.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.874+0000 7f47637fe640 1 --2- 172.21.15.170:0/2999809509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4764148070 0x7f47641d05a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f475c007d50 tx=0x7f475c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.874+0000 7f2689d54640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2684068c80 0x7f26840aaf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.874+0000 7f2689d54640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26840712c0 0x7f26840ab810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.874+0000 7f2689d54640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f26841477e0 0x7f26840abd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.874+0000 7f2689d54640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f26840704f0 con 0x7f26840712c0 2024-05-08T04:08:22.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.874+0000 7f2689d54640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2684070370 con 0x7f2684068c80 2024-05-08T04:08:22.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.874+0000 7f2689d54640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f26840b2ba0 con 0x7f26841477e0 2024-05-08T04:08:22.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.874+0000 7f2682ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26840712c0 0x7f26840ab810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.874+0000 7f2682ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26840712c0 0x7f26840ab810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49136/0 (socket says 172.21.15.170:49136) 2024-05-08T04:08:22.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.874+0000 7f2682ffd640 1 -- 172.21.15.170:0/2139726999 learned_addr learned my addr 172.21.15.170:0/2139726999 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:22.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.874+0000 7f2683fff640 1 --2- 172.21.15.170:0/2139726999 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f26841477e0 0x7f26840abd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.874+0000 7f66cd4ca640 1 --2- 172.21.15.170:0/2611151330 >> v2:172.21.15.187:6804/2815417961 conn(0x7f66b4080780 0x7f66b4082ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.875+0000 7f62b57fa640 1 -- 172.21.15.170:0/154066009 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a8007de0 con 0x7f62b8140780 2024-05-08T04:08:22.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.875+0000 7f26837fe640 1 --2- 172.21.15.170:0/2139726999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2684068c80 0x7f26840aaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.875+0000 7f4acd75b640 1 --2- 172.21.15.170:0/4218083282 >> v2:172.21.15.187:6816/101007897 conn(0x7f4aac076360 0x7f4aac078820 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0037000 tx=0x7f4ac0032000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.875+0000 7feea1ffb640 1 -- 172.21.15.170:0/844936287 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7fee9c01b070 con 0x7feeac070120 2024-05-08T04:08:22.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.875+0000 7f3df27fc640 1 -- 172.21.15.170:0/1966442353 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df8007d70 con 0x7f3dfc0b10d0 2024-05-08T04:08:22.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.875+0000 7f3e03e6f640 1 -- 172.21.15.170:0/1966442353 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3de4000f80 con 0x7f3dfc0b10d0 2024-05-08T04:08:22.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.875+0000 7f475bfff640 1 -- 172.21.15.170:0/2999809509 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c00edd0 con 0x7f4764148070 2024-05-08T04:08:22.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.875+0000 7f475bfff640 1 -- 172.21.15.170:0/2999809509 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f475c003990 con 0x7f4764148070 2024-05-08T04:08:22.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.875+0000 7f475bfff640 1 -- 172.21.15.170:0/2999809509 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475c003d60 con 0x7f4764148070 2024-05-08T04:08:22.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.875+0000 7f66c7fff640 1 --2- 172.21.15.170:0/2611151330 >> v2:172.21.15.187:6816/101007897 conn(0x7f66b407a7a0 0x7f66b407cc60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f66c80b3280 tx=0x7f66c0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.875+0000 7f26837fe640 1 -- 172.21.15.170:0/2139726999 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f26841477e0 msgr2=0x7f26840abd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.875+0000 7f26837fe640 1 --2- 172.21.15.170:0/2139726999 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f26841477e0 0x7f26840abd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.875+0000 7f26837fe640 1 -- 172.21.15.170:0/2139726999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26840712c0 msgr2=0x7f26840ab810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.876+0000 7f66cd4ca640 1 --2- 172.21.15.170:0/2611151330 >> v2:172.21.15.187:6804/2815417961 conn(0x7f66b4080780 0x7f66b4082ba0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.875+0000 7f26837fe640 1 --2- 172.21.15.170:0/2139726999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26840712c0 0x7f26840ab810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.875+0000 7f26837fe640 1 -- 172.21.15.170:0/2139726999 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26840ad060 con 0x7f2684068c80 2024-05-08T04:08:22.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.876+0000 7f2683fff640 1 --2- 172.21.15.170:0/2139726999 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f26841477e0 0x7f26840abd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:08:22.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.875+0000 7fdc97fff640 1 -- 172.21.15.170:0/2187537070 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc98146f00 con 0x7fdc80003a30 2024-05-08T04:08:22.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.875+0000 7fdc97fff640 1 -- 172.21.15.170:0/2187537070 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc98128020 con 0x7fdc80003a30 2024-05-08T04:08:22.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.876+0000 7f26837fe640 1 --2- 172.21.15.170:0/2139726999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2684068c80 0x7f26840aaf50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2674005880 tx=0x7f267400c960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.876+0000 7f62b57fa640 1 -- 172.21.15.170:0/154066009 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f62a8018070 con 0x7f62b8140780 2024-05-08T04:08:22.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.876+0000 7fdc7f7fe640 1 -- 172.21.15.170:0/2187537070 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c037040 con 0x7fdc80003a30 2024-05-08T04:08:22.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.876+0000 7f629effd640 1 -- 172.21.15.170:0/154066009 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6298000f80 con 0x7f62b8140780 2024-05-08T04:08:22.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.876+0000 7fdc97fff640 1 -- 172.21.15.170:0/2187537070 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdc981e0ec0 con 0x7fdc80003a30 2024-05-08T04:08:22.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.876+0000 7f47693ce640 1 -- 172.21.15.170:0/2999809509 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47641cf960 con 0x7f4764148070 2024-05-08T04:08:22.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.876+0000 7feea1ffb640 1 --2- 172.21.15.170:0/844936287 >> v2:172.21.15.187:6816/101007897 conn(0x7fee90076360 0x7fee90078820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.876+0000 7f3df27fc640 1 -- 172.21.15.170:0/1966442353 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f3df4005d10 con 0x7f3dfc0b10d0 2024-05-08T04:08:22.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.876+0000 7f47693ce640 1 -- 172.21.15.170:0/2999809509 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f476408cab0 con 0x7f4764148070 2024-05-08T04:08:22.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.876+0000 7feea1ffb640 1 -- 172.21.15.170:0/844936287 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4090+0+0 (secure 0 0 0) 0x7fee9c096bf0 con 0x7feeac070120 2024-05-08T04:08:22.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.876+0000 7f266bfff640 1 -- 172.21.15.170:0/2139726999 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674017070 con 0x7f2684068c80 2024-05-08T04:08:22.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.876+0000 7f266bfff640 1 -- 172.21.15.170:0/2139726999 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2674005ce0 con 0x7f2684068c80 2024-05-08T04:08:22.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.876+0000 7f266bfff640 1 -- 172.21.15.170:0/2139726999 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26740072d0 con 0x7f2684068c80 2024-05-08T04:08:22.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.876+0000 7f2689d54640 1 -- 172.21.15.170:0/2139726999 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26840ad340 con 0x7f2684068c80 2024-05-08T04:08:22.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.876+0000 7f2689d54640 1 -- 172.21.15.170:0/2139726999 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26841fa680 con 0x7f2684068c80 2024-05-08T04:08:22.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.876+0000 7f4ace75d640 1 --2- 172.21.15.170:0/4218083282 >> v2:172.21.15.187:6808/1425849822 conn(0x7f4aac07c340 0x7f4aac07e760 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.876+0000 7feea1ffb640 1 --2- 172.21.15.170:0/844936287 >> v2:172.21.15.170:6808/3364584304 conn(0x7fee9007c340 0x7fee9007e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.876+0000 7feea1ffb640 1 -- 172.21.15.170:0/844936287 --> v2:172.21.15.170:6808/3364584304 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fee9c01ee80 con 0x7fee9007c340 2024-05-08T04:08:22.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.876+0000 7feea1ffb640 1 -- 172.21.15.170:0/844936287 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fee9c096fe0 con 0x7feeac070120 2024-05-08T04:08:22.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.877+0000 7feeb1c8a640 1 --2- 172.21.15.170:0/844936287 >> v2:172.21.15.170:6808/3364584304 conn(0x7fee9007c340 0x7fee9007e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.877+0000 7f62b57fa640 1 --2- 172.21.15.170:0/154066009 >> v2:172.21.15.187:6816/101007897 conn(0x7f62a4076290 0x7f62a4078750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.877+0000 7f66c57fa640 1 -- 172.21.15.170:0/2611151330 <== osd.7 v2:172.21.15.187:6804/2815417961 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f66b8015e50 con 0x7f66b4080780 2024-05-08T04:08:22.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.877+0000 7f62b57fa640 1 -- 172.21.15.170:0/154066009 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4090+0+0 (secure 0 0 0) 0x7f62a8096fe0 con 0x7f62b8140780 2024-05-08T04:08:22.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.877+0000 7feeb0c88640 1 --2- 172.21.15.170:0/844936287 >> v2:172.21.15.187:6816/101007897 conn(0x7fee90076360 0x7fee90078820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.877+0000 7f62bcd89640 1 --2- 172.21.15.170:0/154066009 >> v2:172.21.15.187:6816/101007897 conn(0x7f62a4076290 0x7f62a4078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.877+0000 7f3df27fc640 1 --2- 172.21.15.170:0/1966442353 >> v2:172.21.15.187:6816/101007897 conn(0x7f3de0076360 0x7f3de0078820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.877+0000 7f62b57fa640 1 --2- 172.21.15.170:0/154066009 >> v2:172.21.15.170:6804/3333432230 conn(0x7f62a407c270 0x7f62a407e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.877+0000 7f62b57fa640 1 -- 172.21.15.170:0/154066009 --> v2:172.21.15.170:6804/3333432230 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f62a407ed00 con 0x7f62a407c270 2024-05-08T04:08:22.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.878+0000 7f62b57fa640 1 -- 172.21.15.170:0/154066009 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f62a8099560 con 0x7f62b8140780 2024-05-08T04:08:22.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.877+0000 7f4abe7fc640 1 -- 172.21.15.170:0/4218083282 <== osd.4 v2:172.21.15.187:6808/1425849822 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4aac07edd0 con 0x7f4aac07c340 2024-05-08T04:08:22.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.878+0000 7f47693ce640 1 -- 172.21.15.170:0/2999809509 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f476406b400 con 0x7f4764148070 2024-05-08T04:08:22.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.877+0000 7feeb1c8a640 1 --2- 172.21.15.170:0/844936287 >> v2:172.21.15.170:6808/3364584304 conn(0x7fee9007c340 0x7fee9007e760 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.878+0000 7f3df27fc640 1 -- 172.21.15.170:0/1966442353 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4090+0+0 (secure 0 0 0) 0x7f3df4097010 con 0x7f3dfc0b10d0 2024-05-08T04:08:22.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.878+0000 7feeb0c88640 1 --2- 172.21.15.170:0/844936287 >> v2:172.21.15.187:6816/101007897 conn(0x7fee90076360 0x7fee90078820 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7feea8005880 tx=0x7feea8007a20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.878+0000 7f3df27fc640 1 --2- 172.21.15.170:0/1966442353 >> v2:172.21.15.187:6800/3273856499 conn(0x7f3de007c340 0x7f3de007e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.878+0000 7f3df27fc640 1 -- 172.21.15.170:0/1966442353 --> v2:172.21.15.187:6800/3273856499 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3df401fe50 con 0x7f3de007c340 2024-05-08T04:08:22.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.878+0000 7f3df27fc640 1 -- 172.21.15.170:0/1966442353 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f3df409e050 con 0x7f3dfc0b10d0 2024-05-08T04:08:22.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.878+0000 7f2689d54640 1 -- 172.21.15.170:0/2139726999 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2664000f80 con 0x7f2684068c80 2024-05-08T04:08:22.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.878+0000 7f3e01c0b640 1 --2- 172.21.15.170:0/1966442353 >> v2:172.21.15.187:6816/101007897 conn(0x7f3de0076360 0x7f3de0078820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.878+0000 7f3e0140a640 1 --2- 172.21.15.170:0/1966442353 >> v2:172.21.15.187:6800/3273856499 conn(0x7f3de007c340 0x7f3de007e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.879+0000 7feea1ffb640 1 -- 172.21.15.170:0/844936287 <== osd.1 v2:172.21.15.170:6808/3364584304 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fee9c01ee80 con 0x7fee9007c340 2024-05-08T04:08:22.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.879+0000 7f62bd58a640 1 --2- 172.21.15.170:0/154066009 >> v2:172.21.15.170:6804/3333432230 conn(0x7f62a407c270 0x7f62a407e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.879+0000 7fdc7f7fe640 1 -- 172.21.15.170:0/2187537070 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7fdc8c044070 con 0x7fdc80003a30 2024-05-08T04:08:22.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.879+0000 7f3e0140a640 1 --2- 172.21.15.170:0/1966442353 >> v2:172.21.15.187:6800/3273856499 conn(0x7f3de007c340 0x7f3de007e760 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.879+0000 7f62bcd89640 1 --2- 172.21.15.170:0/154066009 >> v2:172.21.15.187:6816/101007897 conn(0x7f62a4076290 0x7f62a4078750 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f62ac039000 tx=0x7f62ac034000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.879+0000 7f3e01c0b640 1 --2- 172.21.15.170:0/1966442353 >> v2:172.21.15.187:6816/101007897 conn(0x7f3de0076360 0x7f3de0078820 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3df8001240 tx=0x7f3df80078f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.879+0000 7f266bfff640 1 -- 172.21.15.170:0/2139726999 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f267401a070 con 0x7f2684068c80 2024-05-08T04:08:22.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.880+0000 7f475bfff640 1 -- 172.21.15.170:0/2999809509 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f475c01b020 con 0x7f4764148070 2024-05-08T04:08:22.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.880+0000 7fdc7f7fe640 1 --2- 172.21.15.170:0/2187537070 >> v2:172.21.15.187:6816/101007897 conn(0x7fdc84076290 0x7fdc84078750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.880+0000 7fdc7f7fe640 1 -- 172.21.15.170:0/2187537070 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 4090+0+0 (secure 0 0 0) 0x7fdc8c0b9160 con 0x7fdc80003a30 2024-05-08T04:08:22.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.880+0000 7f62bd58a640 1 --2- 172.21.15.170:0/154066009 >> v2:172.21.15.170:6804/3333432230 conn(0x7f62a407c270 0x7f62a407e690 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.880+0000 7fdc7f7fe640 1 --2- 172.21.15.170:0/2187537070 >> v2:172.21.15.170:6800/1037021465 conn(0x7fdc8407c270 0x7fdc8407e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.880+0000 7fdc7f7fe640 1 -- 172.21.15.170:0/2187537070 --> v2:172.21.15.170:6800/1037021465 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdc8c007e50 con 0x7fdc8407c270 2024-05-08T04:08:22.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.880+0000 7f266bfff640 1 --2- 172.21.15.170:0/2139726999 >> v2:172.21.15.187:6816/101007897 conn(0x7f2670076390 0x7f2670078850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.880+0000 7fdc7f7fe640 1 -- 172.21.15.170:0/2187537070 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7fdc8c0b9550 con 0x7fdc80003a30 2024-05-08T04:08:22.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.880+0000 7f475bfff640 1 --2- 172.21.15.170:0/2999809509 >> v2:172.21.15.187:6816/101007897 conn(0x7f47480762e0 0x7f47480787a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.880+0000 7f2682ffd640 1 --2- 172.21.15.170:0/2139726999 >> v2:172.21.15.187:6816/101007897 conn(0x7f2670076390 0x7f2670078850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.945 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.880+0000 7f475bfff640 1 -- 172.21.15.170:0/2999809509 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4090+0+0 (secure 0 0 0) 0x7f475c097060 con 0x7f4764148070 2024-05-08T04:08:22.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.880+0000 7f266bfff640 1 -- 172.21.15.170:0/2139726999 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4090+0+0 (secure 0 0 0) 0x7f2674096120 con 0x7f2684068c80 2024-05-08T04:08:22.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.880+0000 7f475bfff640 1 --2- 172.21.15.170:0/2999809509 >> v2:172.21.15.187:6809/2911331731 conn(0x7f474807c2c0 0x7f474807e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.880+0000 7f475bfff640 1 -- 172.21.15.170:0/2999809509 --> v2:172.21.15.187:6809/2911331731 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f474807ed50 con 0x7f474807c2c0 2024-05-08T04:08:22.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.880+0000 7f475bfff640 1 -- 172.21.15.170:0/2999809509 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f475c09e050 con 0x7f4764148070 2024-05-08T04:08:22.946 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.880+0000 7f3df27fc640 1 -- 172.21.15.170:0/1966442353 <== osd.5 v2:172.21.15.187:6800/3273856499 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3df401fe50 con 0x7f3de007c340 2024-05-08T04:08:22.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.881+0000 7fdc977fe640 1 --2- 172.21.15.170:0/2187537070 >> v2:172.21.15.170:6800/1037021465 conn(0x7fdc8407c270 0x7fdc8407e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.881+0000 7f2682ffd640 1 --2- 172.21.15.170:0/2139726999 >> v2:172.21.15.187:6816/101007897 conn(0x7f2670076390 0x7f2670078850 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2678009720 tx=0x7f267803d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.881+0000 7f266bfff640 1 --2- 172.21.15.170:0/2139726999 >> v2:172.21.15.170:6812/153555120 conn(0x7f267007c370 0x7f267007e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:22.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.881+0000 7f266bfff640 1 -- 172.21.15.170:0/2139726999 --> v2:172.21.15.170:6812/153555120 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2674015e70 con 0x7f267007c370 2024-05-08T04:08:22.947 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.881+0000 7f266bfff640 1 -- 172.21.15.170:0/2139726999 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f267409d050 con 0x7f2684068c80 2024-05-08T04:08:22.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.881+0000 7f2683fff640 1 --2- 172.21.15.170:0/2139726999 >> v2:172.21.15.170:6812/153555120 conn(0x7f267007c370 0x7f267007e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.881+0000 7fdc967fc640 1 --2- 172.21.15.170:0/2187537070 >> v2:172.21.15.187:6816/101007897 conn(0x7fdc84076290 0x7fdc84078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.881+0000 7f4762ffd640 1 --2- 172.21.15.170:0/2999809509 >> v2:172.21.15.187:6816/101007897 conn(0x7f47480762e0 0x7f47480787a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.881+0000 7f475a7fc640 1 --2- 172.21.15.170:0/2999809509 >> v2:172.21.15.187:6809/2911331731 conn(0x7f474807c2c0 0x7f474807e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:22.948 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.881+0000 7f62b57fa640 1 -- 172.21.15.170:0/154066009 <== osd.2 v2:172.21.15.170:6804/3333432230 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f62a407ed00 con 0x7f62a407c270 2024-05-08T04:08:22.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.881+0000 7f2683fff640 1 --2- 172.21.15.170:0/2139726999 >> v2:172.21.15.170:6812/153555120 conn(0x7f267007c370 0x7f267007e790 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.882+0000 7f4762ffd640 1 --2- 172.21.15.170:0/2999809509 >> v2:172.21.15.187:6816/101007897 conn(0x7f47480762e0 0x7f47480787a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4754036000 tx=0x7f475403d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.882+0000 7fdc977fe640 1 --2- 172.21.15.170:0/2187537070 >> v2:172.21.15.170:6800/1037021465 conn(0x7fdc8407c270 0x7fdc8407e690 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.949 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.882+0000 7f475a7fc640 1 --2- 172.21.15.170:0/2999809509 >> v2:172.21.15.187:6809/2911331731 conn(0x7f474807c2c0 0x7f474807e6e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.882+0000 7fdc967fc640 1 --2- 172.21.15.170:0/2187537070 >> v2:172.21.15.187:6816/101007897 conn(0x7fdc84076290 0x7fdc84078750 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdc980ae060 tx=0x7fdc88008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:22.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.883+0000 7f266bfff640 1 -- 172.21.15.170:0/2139726999 <== osd.3 v2:172.21.15.170:6812/153555120 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2674015e70 con 0x7f267007c370 2024-05-08T04:08:22.950 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.883+0000 7f475bfff640 1 -- 172.21.15.170:0/2999809509 <== osd.6 v2:172.21.15.187:6809/2911331731 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f474807ed50 con 0x7f474807c2c0 2024-05-08T04:08:22.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.883+0000 7fdc7f7fe640 1 -- 172.21.15.170:0/2187537070 <== osd.0 v2:172.21.15.170:6800/1037021465 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdc8c007e50 con 0x7fdc8407c270 2024-05-08T04:08:22.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.908+0000 7f66cef2d640 1 -- 172.21.15.170:0/2611151330 --> v2:172.21.15.187:6804/2815417961 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f66c806c7d0 con 0x7f66b4080780 2024-05-08T04:08:22.951 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.910+0000 7f66c57fa640 1 -- 172.21.15.170:0/2611151330 <== osd.7 v2:172.21.15.187:6804/2815417961 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f66c806c7d0 con 0x7f66b4080780 2024-05-08T04:08:22.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.910+0000 7f66aeffd640 1 -- 172.21.15.170:0/2611151330 >> v2:172.21.15.187:6804/2815417961 conn(0x7f66b4080780 msgr2=0x7f66b4082ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.910+0000 7f66aeffd640 1 --2- 172.21.15.170:0/2611151330 >> v2:172.21.15.187:6804/2815417961 conn(0x7f66b4080780 0x7f66b4082ba0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.910+0000 7f66aeffd640 1 -- 172.21.15.170:0/2611151330 >> v2:172.21.15.187:6816/101007897 conn(0x7f66b407a7a0 msgr2=0x7f66b407cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.911+0000 7f66aeffd640 1 --2- 172.21.15.170:0/2611151330 >> v2:172.21.15.187:6816/101007897 conn(0x7f66b407a7a0 0x7f66b407cc60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f66c80b3280 tx=0x7f66c0009040 comp rx=0 tx=0).stop 2024-05-08T04:08:22.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.911+0000 7f66aeffd640 1 -- 172.21.15.170:0/2611151330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66c8070320 msgr2=0x7f66c80ae4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.911+0000 7f66aeffd640 1 --2- 172.21.15.170:0/2611151330 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66c8070320 0x7f66c80ae4a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f66b800c8d0 tx=0x7f66b800cd90 comp rx=0 tx=0).stop 2024-05-08T04:08:22.952 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.911+0000 7f66cd4ca640 1 -- 172.21.15.170:0/2611151330 reap_dead start 2024-05-08T04:08:22.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.911+0000 7fee977fe640 1 -- 172.21.15.170:0/844936287 --> v2:172.21.15.170:6808/3364584304 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7feeac073630 con 0x7fee9007c340 2024-05-08T04:08:22.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.911+0000 7f66aeffd640 1 -- 172.21.15.170:0/2611151330 shutdown_connections 2024-05-08T04:08:22.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.911+0000 7f4acef5e640 1 -- 172.21.15.170:0/4218083282 --> v2:172.21.15.187:6808/1425849822 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4ab0002c70 con 0x7f4aac07c340 2024-05-08T04:08:22.953 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.911+0000 7f66aeffd640 1 -- 172.21.15.170:0/2611151330 >> 172.21.15.170:0/2611151330 conn(0x7f66c80b4e30 msgr2=0x7f66c80b5210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:22.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.912+0000 7feea1ffb640 1 -- 172.21.15.170:0/844936287 <== osd.1 v2:172.21.15.170:6808/3364584304 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7feeac073630 con 0x7fee9007c340 2024-05-08T04:08:22.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.912+0000 7f3e03e6f640 1 -- 172.21.15.170:0/1966442353 --> v2:172.21.15.187:6800/3273856499 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3de4002c70 con 0x7f3de007c340 2024-05-08T04:08:22.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.912+0000 7f66aeffd640 1 -- 172.21.15.170:0/2611151330 shutdown_connections 2024-05-08T04:08:22.954 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.912+0000 7f66aeffd640 1 -- 172.21.15.170:0/2611151330 wait complete. 2024-05-08T04:08:22.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.912+0000 7f4abe7fc640 1 -- 172.21.15.170:0/4218083282 <== osd.4 v2:172.21.15.187:6808/1425849822 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4ab0002c70 con 0x7f4aac07c340 2024-05-08T04:08:22.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.912+0000 7fee977fe640 1 -- 172.21.15.170:0/844936287 >> v2:172.21.15.170:6808/3364584304 conn(0x7fee9007c340 msgr2=0x7fee9007e760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.912+0000 7fee977fe640 1 --2- 172.21.15.170:0/844936287 >> v2:172.21.15.170:6808/3364584304 conn(0x7fee9007c340 0x7fee9007e760 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.955 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.912+0000 7f3df27fc640 1 -- 172.21.15.170:0/1966442353 <== osd.5 v2:172.21.15.187:6800/3273856499 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3de4002c70 con 0x7f3de007c340 2024-05-08T04:08:22.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.912+0000 7f4acef5e640 1 -- 172.21.15.170:0/4218083282 >> v2:172.21.15.187:6808/1425849822 conn(0x7f4aac07c340 msgr2=0x7f4aac07e760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.912+0000 7f4acef5e640 1 --2- 172.21.15.170:0/4218083282 >> v2:172.21.15.187:6808/1425849822 conn(0x7f4aac07c340 0x7f4aac07e760 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.912+0000 7fee977fe640 1 -- 172.21.15.170:0/844936287 >> v2:172.21.15.187:6816/101007897 conn(0x7fee90076360 msgr2=0x7fee90078820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.956 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.913+0000 7fee977fe640 1 --2- 172.21.15.170:0/844936287 >> v2:172.21.15.187:6816/101007897 conn(0x7fee90076360 0x7fee90078820 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7feea8005880 tx=0x7feea8007a20 comp rx=0 tx=0).stop 2024-05-08T04:08:22.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.913+0000 7fee977fe640 1 -- 172.21.15.170:0/844936287 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feeac070120 msgr2=0x7feeac147c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.913+0000 7fee977fe640 1 --2- 172.21.15.170:0/844936287 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feeac070120 0x7feeac147c30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00c7b0 tx=0x7fee9c00cc70 comp rx=0 tx=0).stop 2024-05-08T04:08:22.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.913+0000 7feeb1c8a640 1 -- 172.21.15.170:0/844936287 reap_dead start 2024-05-08T04:08:22.957 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.913+0000 7f2689d54640 1 -- 172.21.15.170:0/2139726999 --> v2:172.21.15.170:6812/153555120 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2664002c70 con 0x7f267007c370 2024-05-08T04:08:22.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.913+0000 7f4acef5e640 1 -- 172.21.15.170:0/4218083282 >> v2:172.21.15.187:6816/101007897 conn(0x7f4aac076360 msgr2=0x7f4aac078820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.913+0000 7f4acef5e640 1 --2- 172.21.15.170:0/4218083282 >> v2:172.21.15.187:6816/101007897 conn(0x7f4aac076360 0x7f4aac078820 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0037000 tx=0x7f4ac0032000 comp rx=0 tx=0).stop 2024-05-08T04:08:22.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.913+0000 7f4acef5e640 1 -- 172.21.15.170:0/4218083282 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4ac80700a0 msgr2=0x7f4ac81c8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.913+0000 7f4acef5e640 1 --2- 172.21.15.170:0/4218083282 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4ac80700a0 0x7f4ac81c8dc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4ac400c760 tx=0x7f4ac400cc20 comp rx=0 tx=0).stop 2024-05-08T04:08:22.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.913+0000 7f4ace75d640 1 -- 172.21.15.170:0/4218083282 reap_dead start 2024-05-08T04:08:22.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.913+0000 7fee977fe640 1 -- 172.21.15.170:0/844936287 shutdown_connections 2024-05-08T04:08:22.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.913+0000 7f3ddffff640 1 -- 172.21.15.170:0/1966442353 >> v2:172.21.15.187:6800/3273856499 conn(0x7f3de007c340 msgr2=0x7f3de007e760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.958 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.913+0000 7f3ddffff640 1 --2- 172.21.15.170:0/1966442353 >> v2:172.21.15.187:6800/3273856499 conn(0x7f3de007c340 0x7f3de007e760 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.913+0000 7fee977fe640 1 -- 172.21.15.170:0/844936287 >> 172.21.15.170:0/844936287 conn(0x7feeac0b4e30 msgr2=0x7feeac0afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:22.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.913+0000 7fee977fe640 1 -- 172.21.15.170:0/844936287 shutdown_connections 2024-05-08T04:08:22.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.913+0000 7fee977fe640 1 -- 172.21.15.170:0/844936287 wait complete. 2024-05-08T04:08:22.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.913+0000 7f4acef5e640 1 -- 172.21.15.170:0/4218083282 shutdown_connections 2024-05-08T04:08:22.959 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.913+0000 7f4acef5e640 1 -- 172.21.15.170:0/4218083282 >> 172.21.15.170:0/4218083282 conn(0x7f4ac80b4bb0 msgr2=0x7f4ac8068180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:22.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.913+0000 7f3ddffff640 1 -- 172.21.15.170:0/1966442353 >> v2:172.21.15.187:6816/101007897 conn(0x7f3de0076360 msgr2=0x7f3de0078820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.913+0000 7f3ddffff640 1 --2- 172.21.15.170:0/1966442353 >> v2:172.21.15.187:6816/101007897 conn(0x7f3de0076360 0x7f3de0078820 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3df8001240 tx=0x7f3df80078f0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.913+0000 7f3ddffff640 1 -- 172.21.15.170:0/1966442353 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3dfc0b10d0 msgr2=0x7f3dfc1d8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.913+0000 7f3ddffff640 1 --2- 172.21.15.170:0/1966442353 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3dfc0b10d0 0x7f3dfc1d8670 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc0745c0 tx=0x7f3df400f4a0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.913+0000 7f4acef5e640 1 -- 172.21.15.170:0/4218083282 shutdown_connections 2024-05-08T04:08:22.960 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.913+0000 7f4acef5e640 1 -- 172.21.15.170:0/4218083282 wait complete. 2024-05-08T04:08:22.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.913+0000 7f266bfff640 1 -- 172.21.15.170:0/2139726999 <== osd.3 v2:172.21.15.170:6812/153555120 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2664002c70 con 0x7f267007c370 2024-05-08T04:08:22.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.913+0000 7f3e0240c640 1 -- 172.21.15.170:0/1966442353 reap_dead start 2024-05-08T04:08:22.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.914+0000 7f3ddffff640 1 -- 172.21.15.170:0/1966442353 shutdown_connections 2024-05-08T04:08:22.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.914+0000 7f3ddffff640 1 -- 172.21.15.170:0/1966442353 >> 172.21.15.170:0/1966442353 conn(0x7f3dfc076050 msgr2=0x7f3dfc140810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:22.961 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.914+0000 7f3ddffff640 1 -- 172.21.15.170:0/1966442353 shutdown_connections 2024-05-08T04:08:22.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.914+0000 7f2689d54640 1 -- 172.21.15.170:0/2139726999 >> v2:172.21.15.170:6812/153555120 conn(0x7f267007c370 msgr2=0x7f267007e790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.914+0000 7f2689d54640 1 --2- 172.21.15.170:0/2139726999 >> v2:172.21.15.170:6812/153555120 conn(0x7f267007c370 0x7f267007e790 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.914+0000 7f3ddffff640 1 -- 172.21.15.170:0/1966442353 wait complete. 2024-05-08T04:08:22.962 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.915+0000 7f629effd640 1 -- 172.21.15.170:0/154066009 --> v2:172.21.15.170:6804/3333432230 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6298002c70 con 0x7f62a407c270 2024-05-08T04:08:22.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.915+0000 7fdc97fff640 1 -- 172.21.15.170:0/2187537070 --> v2:172.21.15.170:6800/1037021465 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdc980696e0 con 0x7fdc8407c270 2024-05-08T04:08:22.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.914+0000 7f2689d54640 1 -- 172.21.15.170:0/2139726999 >> v2:172.21.15.187:6816/101007897 conn(0x7f2670076390 msgr2=0x7f2670078850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.914+0000 7f2689d54640 1 --2- 172.21.15.170:0/2139726999 >> v2:172.21.15.187:6816/101007897 conn(0x7f2670076390 0x7f2670078850 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2678009720 tx=0x7f267803d040 comp rx=0 tx=0).stop 2024-05-08T04:08:22.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.914+0000 7f2689d54640 1 -- 172.21.15.170:0/2139726999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2684068c80 msgr2=0x7f26840aaf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.914+0000 7f2689d54640 1 --2- 172.21.15.170:0/2139726999 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2684068c80 0x7f26840aaf50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2674005880 tx=0x7f267400c960 comp rx=0 tx=0).stop 2024-05-08T04:08:22.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.914+0000 7f2683fff640 1 -- 172.21.15.170:0/2139726999 reap_dead start 2024-05-08T04:08:22.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.915+0000 7f2689d54640 1 -- 172.21.15.170:0/2139726999 shutdown_connections 2024-05-08T04:08:22.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.915+0000 7f2689d54640 1 -- 172.21.15.170:0/2139726999 >> 172.21.15.170:0/2139726999 conn(0x7f26840b5c20 msgr2=0x7f2684145120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:22.963 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.915+0000 7f47693ce640 1 -- 172.21.15.170:0/2999809509 --> v2:172.21.15.187:6809/2911331731 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4764070c80 con 0x7f474807c2c0 2024-05-08T04:08:22.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.915+0000 7f2689d54640 1 -- 172.21.15.170:0/2139726999 shutdown_connections 2024-05-08T04:08:22.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.915+0000 7f62b57fa640 1 -- 172.21.15.170:0/154066009 <== osd.2 v2:172.21.15.170:6804/3333432230 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6298002c70 con 0x7f62a407c270 2024-05-08T04:08:22.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.915+0000 7f2689d54640 1 -- 172.21.15.170:0/2139726999 wait complete. 2024-05-08T04:08:22.964 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.915+0000 7f475bfff640 1 -- 172.21.15.170:0/2999809509 <== osd.6 v2:172.21.15.187:6809/2911331731 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4764070c80 con 0x7f474807c2c0 2024-05-08T04:08:22.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.915+0000 7fdc7f7fe640 1 -- 172.21.15.170:0/2187537070 <== osd.0 v2:172.21.15.170:6800/1037021465 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdc980696e0 con 0x7fdc8407c270 2024-05-08T04:08:22.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.916+0000 7f47693ce640 1 -- 172.21.15.170:0/2999809509 >> v2:172.21.15.187:6809/2911331731 conn(0x7f474807c2c0 msgr2=0x7f474807e6e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.916+0000 7f47693ce640 1 --2- 172.21.15.170:0/2999809509 >> v2:172.21.15.187:6809/2911331731 conn(0x7f474807c2c0 0x7f474807e6e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.916+0000 7f47693ce640 1 -- 172.21.15.170:0/2999809509 >> v2:172.21.15.187:6816/101007897 conn(0x7f47480762e0 msgr2=0x7f47480787a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.965 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.916+0000 7f47693ce640 1 --2- 172.21.15.170:0/2999809509 >> v2:172.21.15.187:6816/101007897 conn(0x7f47480762e0 0x7f47480787a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4754036000 tx=0x7f475403d040 comp rx=0 tx=0).stop 2024-05-08T04:08:22.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.916+0000 7fdc97fff640 1 -- 172.21.15.170:0/2187537070 >> v2:172.21.15.170:6800/1037021465 conn(0x7fdc8407c270 msgr2=0x7fdc8407e690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.916+0000 7fdc97fff640 1 --2- 172.21.15.170:0/2187537070 >> v2:172.21.15.170:6800/1037021465 conn(0x7fdc8407c270 0x7fdc8407e690 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.916+0000 7f47693ce640 1 -- 172.21.15.170:0/2999809509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4764148070 msgr2=0x7f47641d05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.916+0000 7f47693ce640 1 --2- 172.21.15.170:0/2999809509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4764148070 0x7f47641d05a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f475c007d50 tx=0x7f475c00e4b0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.966 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.916+0000 7f47637fe640 1 -- 172.21.15.170:0/2999809509 reap_dead start 2024-05-08T04:08:22.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.916+0000 7fdc97fff640 1 -- 172.21.15.170:0/2187537070 >> v2:172.21.15.187:6816/101007897 conn(0x7fdc84076290 msgr2=0x7fdc84078750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.916+0000 7fdc97fff640 1 --2- 172.21.15.170:0/2187537070 >> v2:172.21.15.187:6816/101007897 conn(0x7fdc84076290 0x7fdc84078750 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdc980ae060 tx=0x7fdc88008040 comp rx=0 tx=0).stop 2024-05-08T04:08:22.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.916+0000 7fdc97fff640 1 -- 172.21.15.170:0/2187537070 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdc80003a30 msgr2=0x7fdc98146460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.916+0000 7fdc97fff640 1 --2- 172.21.15.170:0/2187537070 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdc80003a30 0x7fdc98146460 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdc8000c110 tx=0x7fdc8c0046b0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.916+0000 7fdc977fe640 1 -- 172.21.15.170:0/2187537070 reap_dead start 2024-05-08T04:08:22.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.917+0000 7f629effd640 1 -- 172.21.15.170:0/154066009 >> v2:172.21.15.170:6804/3333432230 conn(0x7f62a407c270 msgr2=0x7f62a407e690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.967 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.917+0000 7f629effd640 1 --2- 172.21.15.170:0/154066009 >> v2:172.21.15.170:6804/3333432230 conn(0x7f62a407c270 0x7f62a407e690 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.916+0000 7f47693ce640 1 -- 172.21.15.170:0/2999809509 shutdown_connections 2024-05-08T04:08:22.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.916+0000 7f47693ce640 1 -- 172.21.15.170:0/2999809509 >> 172.21.15.170:0/2999809509 conn(0x7f47640b5250 msgr2=0x7f4764068180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:22.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.917+0000 7f47693ce640 1 -- 172.21.15.170:0/2999809509 shutdown_connections 2024-05-08T04:08:22.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.917+0000 7f47693ce640 1 -- 172.21.15.170:0/2999809509 wait complete. 2024-05-08T04:08:22.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.917+0000 7fdc97fff640 1 -- 172.21.15.170:0/2187537070 shutdown_connections 2024-05-08T04:08:22.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.917+0000 7fdc97fff640 1 -- 172.21.15.170:0/2187537070 >> 172.21.15.170:0/2187537070 conn(0x7fdc980b4bb0 msgr2=0x7fdc98072d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:22.968 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.917+0000 7fdc97fff640 1 -- 172.21.15.170:0/2187537070 shutdown_connections 2024-05-08T04:08:22.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.917+0000 7fdc97fff640 1 -- 172.21.15.170:0/2187537070 wait complete. 2024-05-08T04:08:22.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.917+0000 7f629effd640 1 -- 172.21.15.170:0/154066009 >> v2:172.21.15.187:6816/101007897 conn(0x7f62a4076290 msgr2=0x7f62a4078750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.918+0000 7f629effd640 1 --2- 172.21.15.170:0/154066009 >> v2:172.21.15.187:6816/101007897 conn(0x7f62a4076290 0x7f62a4078750 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f62ac039000 tx=0x7f62ac034000 comp rx=0 tx=0).stop 2024-05-08T04:08:22.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.918+0000 7f629effd640 1 -- 172.21.15.170:0/154066009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62b8140780 msgr2=0x7f62b80b3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:22.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.918+0000 7f629effd640 1 --2- 172.21.15.170:0/154066009 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f62b8140780 0x7f62b80b3d80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f62a800c7f0 tx=0x7f62a800cbf0 comp rx=0 tx=0).stop 2024-05-08T04:08:22.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.918+0000 7f62bd58a640 1 -- 172.21.15.170:0/154066009 reap_dead start 2024-05-08T04:08:22.969 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.918+0000 7f629effd640 1 -- 172.21.15.170:0/154066009 shutdown_connections 2024-05-08T04:08:22.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.918+0000 7f629effd640 1 -- 172.21.15.170:0/154066009 >> 172.21.15.170:0/154066009 conn(0x7f62b80b5dd0 msgr2=0x7f62b813abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:22.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.918+0000 7f629effd640 1 -- 172.21.15.170:0/154066009 shutdown_connections 2024-05-08T04:08:22.970 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:22.918+0000 7f629effd640 1 -- 172.21.15.170:0/154066009 wait complete. 2024-05-08T04:08:22.970 INFO:teuthology.orchestra.run.smithi170.stdout:60129542147 2024-05-08T04:08:22.971 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-08T04:08:22.972 INFO:teuthology.orchestra.run.smithi170.stdout:60129542147 2024-05-08T04:08:22.972 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T04:08:22.972 INFO:teuthology.orchestra.run.smithi170.stdout:60129542147 2024-05-08T04:08:22.973 INFO:teuthology.orchestra.run.smithi170.stdout:60129542147 2024-05-08T04:08:22.973 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T04:08:22.973 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T04:08:22.974 INFO:teuthology.orchestra.run.smithi170.stdout:60129542147 2024-05-08T04:08:22.975 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T04:08:22.975 INFO:teuthology.orchestra.run.smithi170.stdout:60129542147 2024-05-08T04:08:22.975 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-08T04:08:22.976 INFO:teuthology.orchestra.run.smithi170.stdout:60129542147 2024-05-08T04:08:22.977 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T04:08:22.977 INFO:teuthology.orchestra.run.smithi170.stdout:55834574852 2024-05-08T04:08:22.978 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T04:08:23.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.180+0000 7f4db22db640 1 Processor -- start 2024-05-08T04:08:23.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.178+0000 7f8ba5c45640 1 Processor -- start 2024-05-08T04:08:23.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.178+0000 7f8ba5c45640 1 -- start start 2024-05-08T04:08:23.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.179+0000 7f8ba5c45640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8b980338f0 0x7f8b980cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.179+0000 7f8ba5c45640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b980329c0 con 0x7f8b98007190 2024-05-08T04:08:23.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.179+0000 7f8ba5c45640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b98032c30 con 0x7f8b980d0140 2024-05-08T04:08:23.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.179+0000 7f8ba5c45640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f8b98007570 con 0x7f8b980338f0 2024-05-08T04:08:23.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.179+0000 7f8b9f7fe640 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8b98007190 0x7f8b98009260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:55954/0 (socket says 172.21.15.170:55954) 2024-05-08T04:08:23.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.179+0000 7f8b9f7fe640 1 -- 172.21.15.170:0/376684602 learned_addr learned my addr 172.21.15.170:0/376684602 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-05-08T04:08:23.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.179+0000 7f8b9e7fc640 1 -- 172.21.15.170:0/376684602 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4153023915 0 0) 0x7f8b98032c30 con 0x7f8b980d0140 2024-05-08T04:08:23.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.179+0000 7f8b9e7fc640 1 -- 172.21.15.170:0/376684602 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b90003520 con 0x7f8b980d0140 2024-05-08T04:08:23.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.180+0000 7f8b9e7fc640 1 -- 172.21.15.170:0/376684602 <== mon.1 v1:172.21.15.187:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2102807842 0 0) 0x7f8b90003520 con 0x7f8b980d0140 2024-05-08T04:08:23.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.180+0000 7f8b9e7fc640 1 -- 172.21.15.170:0/376684602 >> v2:172.21.15.187:3300/0 conn(0x7f8b980338f0 msgr2=0x7f8b980cfc00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T04:08:23.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.180+0000 7f8b9e7fc640 1 --2- 172.21.15.170:0/376684602 >> v2:172.21.15.187:3300/0 conn(0x7f8b980338f0 0x7f8b980cfc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.183 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.180+0000 7f8b9e7fc640 1 -- 172.21.15.170:0/376684602 >> v1:172.21.15.170:6789/0 conn(0x7f8b98007190 legacy=0x7f8b98009260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.180+0000 7f8b9e7fc640 1 -- 172.21.15.170:0/376684602 --> v1:172.21.15.187:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b98032f10 con 0x7f8b980d0140 2024-05-08T04:08:23.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.180+0000 7f8b9e7fc640 1 -- 172.21.15.170:0/376684602 <== mon.1 v1:172.21.15.187:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1210551688 0 0) 0x7f8ba0064010 con 0x7f8b980d0140 2024-05-08T04:08:23.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.180+0000 7f8b9e7fc640 1 -- 172.21.15.170:0/376684602 >> v1:172.21.15.187:6789/0 conn(0x7f8b980d0140 legacy=0x7f8b980d6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.180+0000 7f8b9e7fc640 1 --2- 172.21.15.170:0/376684602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b900039b0 0x7f8b90003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.180+0000 7f8b9e7fc640 1 --2- 172.21.15.170:0/376684602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b90004800 0x7f8b90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.180+0000 7f8b9e7fc640 1 --2- 172.21.15.170:0/376684602 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8b900051c0 0x7f8b9000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.180+0000 7f8b9e7fc640 1 -- 172.21.15.170:0/376684602 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8b98007570 con 0x7f8b900039b0 2024-05-08T04:08:23.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.180+0000 7f8b9e7fc640 1 -- 172.21.15.170:0/376684602 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9000c160 con 0x7f8b90004800 2024-05-08T04:08:23.184 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.180+0000 7f8b9e7fc640 1 -- 172.21.15.170:0/376684602 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f8b9000c360 con 0x7f8b900051c0 2024-05-08T04:08:23.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.180+0000 7f4db22db640 1 -- start start 2024-05-08T04:08:23.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.181+0000 7f4db22db640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4dac068c80 0x7f4dac1470d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.181+0000 7f4db22db640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4dac147610 0x7f4dac070d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.181+0000 7f4db22db640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4dac0712c0 0x7f4dac0b1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.181+0000 7f4db22db640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7f4dac069060 con 0x7f4dac068c80 2024-05-08T04:08:23.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.181+0000 7f4db22db640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f4dac14b1e0 con 0x7f4dac0712c0 2024-05-08T04:08:23.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.181+0000 7f4db22db640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f4dac0afab0 con 0x7f4dac147610 2024-05-08T04:08:23.185 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.181+0000 7f4dab7fe640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4dac147610 0x7f4dac070d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.181+0000 7f7c4f31a640 1 Processor -- start 2024-05-08T04:08:23.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.181+0000 7f7c4f31a640 1 -- start start 2024-05-08T04:08:23.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.181+0000 7f4dab7fe640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4dac147610 0x7f4dac070d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:54380/0 (socket says 172.21.15.170:54380) 2024-05-08T04:08:23.186 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.181+0000 7f4dab7fe640 1 -- 172.21.15.170:0/3053786822 learned_addr learned my addr 172.21.15.170:0/3053786822 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:23.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.180+0000 7f8b9ffff640 1 --2- 172.21.15.170:0/376684602 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8b900051c0 0x7f8b9000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.181+0000 7f8b9ffff640 1 -- 172.21.15.170:0/376684602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b90004800 msgr2=0x7f8b90004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T04:08:23.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.181+0000 7f8b9ffff640 1 --2- 172.21.15.170:0/376684602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b90004800 0x7f8b90004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.181+0000 7f8b9ffff640 1 -- 172.21.15.170:0/376684602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b900039b0 msgr2=0x7f8b90003dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T04:08:23.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.181+0000 7f8b9ffff640 1 --2- 172.21.15.170:0/376684602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b900039b0 0x7f8b90003dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.181+0000 7f8b9ffff640 1 -- 172.21.15.170:0/376684602 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b98032f10 con 0x7f8b900051c0 2024-05-08T04:08:23.187 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.180+0000 7f12b1fce640 1 Processor -- start 2024-05-08T04:08:23.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.180+0000 7f12b1fce640 1 -- start start 2024-05-08T04:08:23.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.181+0000 7f12b1fce640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f12ac1454c0 0x7f12ac070500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.181+0000 7f12b1fce640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f12ac14b3b0 0x7f12ac14b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.181+0000 7f12b1fce640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12ac072370 con 0x7f12ac070120 2024-05-08T04:08:23.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.181+0000 7f12b1fce640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f12ac072650 con 0x7f12ac1454c0 2024-05-08T04:08:23.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.181+0000 7f12b1fce640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f12ac0714c0 con 0x7f12ac14b3b0 2024-05-08T04:08:23.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.182+0000 7f4db0878640 1 --2- 172.21.15.170:0/3053786822 >> v2:172.21.15.187:3300/0 conn(0x7f4dac0712c0 0x7f4dac0b1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.188 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.182+0000 7f4dabfff640 1 --2- 172.21.15.170:0/3053786822 >> v2:172.21.15.170:3300/0 conn(0x7f4dac068c80 0x7f4dac1470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.182+0000 7f8b9ffff640 1 -- 172.21.15.170:0/376684602 reap_dead start 2024-05-08T04:08:23.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.182+0000 7f4dabfff640 1 -- 172.21.15.170:0/3053786822 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4dac147610 msgr2=0x7f4dac070d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:23.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.182+0000 7f4dabfff640 1 --2- 172.21.15.170:0/3053786822 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4dac147610 0x7f4dac070d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.182+0000 7f4dabfff640 1 -- 172.21.15.170:0/3053786822 >> v2:172.21.15.187:3300/0 conn(0x7f4dac0712c0 msgr2=0x7f4dac0b1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:23.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.182+0000 7f4dabfff640 1 --2- 172.21.15.170:0/3053786822 >> v2:172.21.15.187:3300/0 conn(0x7f4dac0712c0 0x7f4dac0b1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.189 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.182+0000 7f4dabfff640 1 -- 172.21.15.170:0/3053786822 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dac147a50 con 0x7f4dac068c80 2024-05-08T04:08:23.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.182+0000 7f12ab7fe640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f12ac070120 0x7f12ac144db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.170:50324/0 (socket says 172.21.15.170:50324) 2024-05-08T04:08:23.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.182+0000 7f12ab7fe640 1 -- 172.21.15.170:0/969768800 learned_addr learned my addr 172.21.15.170:0/969768800 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-05-08T04:08:23.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.182+0000 7f12abfff640 1 --2- 172.21.15.170:0/969768800 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f12ac14b3b0 0x7f12ac14b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.181+0000 7f7d76117640 1 Processor -- start 2024-05-08T04:08:23.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.182+0000 7f7d76117640 1 -- start start 2024-05-08T04:08:23.190 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.182+0000 7f7c4f31a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7c480712b0 0x7f7c48073ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.182+0000 7f12aaffd640 1 --2- 172.21.15.170:0/969768800 >> v2:172.21.15.187:3300/0 conn(0x7f12ac1454c0 0x7f12ac070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.182+0000 7f7c4f31a640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c48073120 con 0x7f7c48070ed0 2024-05-08T04:08:23.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.182+0000 7f7c4f31a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c48073400 con 0x7f7c48074010 2024-05-08T04:08:23.191 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.182+0000 7f7c4f31a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f7c480721f0 con 0x7f7c480712b0 2024-05-08T04:08:23.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.183+0000 7f7d76117640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7d680e2e50 0x7f7d680e5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.183+0000 7f7d76117640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7d680e5ec0 0x7f7d680ea0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.183+0000 7f7d76117640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d6802f8e0 con 0x7f7d68032e80 2024-05-08T04:08:23.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.183+0000 7f7c4eb19640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f7c48074010 0x7f7c4814ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.170:50336/0 (socket says 172.21.15.170:50336) 2024-05-08T04:08:23.192 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.183+0000 7f7c4eb19640 1 -- 172.21.15.170:0/1512319638 learned_addr learned my addr 172.21.15.170:0/1512319638 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-05-08T04:08:23.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.183+0000 7f7d76117640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7f7d6802fbc0 con 0x7f7d680e2e50 2024-05-08T04:08:23.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.183+0000 7f7d76117640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f7d6802d700 con 0x7f7d680e5ec0 2024-05-08T04:08:23.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.183+0000 7f7c4db17640 1 --2- 172.21.15.170:0/1512319638 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7c480712b0 0x7f7c48073ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.183+0000 7f427b59e640 1 Processor -- start 2024-05-08T04:08:23.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.183+0000 7f7c4d316640 1 -- 172.21.15.170:0/1512319638 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1599712680 0 0) 0x7f7c48073400 con 0x7f7c48074010 2024-05-08T04:08:23.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.183+0000 7f7c4d316640 1 -- 172.21.15.170:0/1512319638 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c34003520 con 0x7f7c48074010 2024-05-08T04:08:23.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.183+0000 7f427b59e640 1 -- start start 2024-05-08T04:08:23.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.182+0000 7f8b9ffff640 1 --2- 172.21.15.170:0/376684602 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8b900051c0 0x7f8b9000b9c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0075170 tx=0x7f8b90005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92d341568b8288a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:23.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.184+0000 7f8b9e7fc640 1 -- 172.21.15.170:0/376684602 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0077440 con 0x7f8b900051c0 2024-05-08T04:08:23.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.184+0000 7f8b9e7fc640 1 -- 172.21.15.170:0/376684602 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ba00700d0 con 0x7f8b900051c0 2024-05-08T04:08:23.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.184+0000 7f8b9e7fc640 1 -- 172.21.15.170:0/376684602 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba006e6b0 con 0x7f8b900051c0 2024-05-08T04:08:23.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.184+0000 7f7d74914640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7d680e2e50 0x7f7d680e5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.184+0000 7f7d74914640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7d680e2e50 0x7f7d680e5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49156/0 (socket says 172.21.15.170:49156) 2024-05-08T04:08:23.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.184+0000 7f7d74914640 1 -- 172.21.15.170:0/323260053 learned_addr learned my addr 172.21.15.170:0/323260053 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:23.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.184+0000 7f4dabfff640 1 --2- 172.21.15.170:0/3053786822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4dac068c80 0x7f4dac1470d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4dac06e8a0 tx=0x7f4da402bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8783327d4ec4e5ad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:23.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.184+0000 7f7d75916640 1 --2- 172.21.15.170:0/323260053 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7d680e5ec0 0x7f7d680ea0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.183+0000 7f12aa7fc640 1 -- 172.21.15.170:0/969768800 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1746693825 0 0) 0x7f12ac072370 con 0x7f12ac070120 2024-05-08T04:08:23.196 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.183+0000 7f12aa7fc640 1 -- 172.21.15.170:0/969768800 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1294003520 con 0x7f12ac070120 2024-05-08T04:08:23.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.183+0000 7f7c4db17640 1 -- 172.21.15.170:0/1512319638 >> v1:172.21.15.187:6789/0 conn(0x7f7c48074010 legacy=0x7f7c4814ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.183+0000 7f7c4db17640 1 -- 172.21.15.170:0/1512319638 >> v1:172.21.15.170:6789/0 conn(0x7f7c48070ed0 legacy=0x7f7c48140900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:23.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.184+0000 7f4daaffd640 1 -- 172.21.15.170:0/3053786822 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da4039070 con 0x7f4dac068c80 2024-05-08T04:08:23.197 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.184+0000 7f427b59e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f427c0700a0 0x7f427c070de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.183+0000 7f7c4db17640 1 -- 172.21.15.170:0/1512319638 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c480b0380 con 0x7f7c480712b0 2024-05-08T04:08:23.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.184+0000 7f8ba5c45640 1 -- 172.21.15.170:0/376684602 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8b900051c0 msgr2=0x7f8b9000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.184+0000 7f8ba5c45640 1 --2- 172.21.15.170:0/376684602 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8b900051c0 0x7f8b9000b9c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0075170 tx=0x7f8b90005830 comp rx=0 tx=0).stop 2024-05-08T04:08:23.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.183+0000 7f12abfff640 1 -- 172.21.15.170:0/969768800 >> v2:172.21.15.187:3300/0 conn(0x7f12ac1454c0 msgr2=0x7f12ac070500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:23.198 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.183+0000 7f12abfff640 1 --2- 172.21.15.170:0/969768800 >> v2:172.21.15.187:3300/0 conn(0x7f12ac1454c0 0x7f12ac070500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.183+0000 7f12abfff640 1 -- 172.21.15.170:0/969768800 >> v1:172.21.15.187:6789/0 conn(0x7f12ac070120 legacy=0x7f12ac144db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.183+0000 7f12abfff640 1 -- 172.21.15.170:0/969768800 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12ac071150 con 0x7f12ac14b3b0 2024-05-08T04:08:23.199 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.184+0000 7f12abfff640 1 --2- 172.21.15.170:0/969768800 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f12ac14b3b0 0x7f12ac14b800 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f129c0097a0 tx=0x7f129c02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cc0a66ea4dbaab4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:23.201 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.185+0000 7f4daaffd640 1 -- 172.21.15.170:0/3053786822 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4da402cb40 con 0x7f4dac068c80 2024-05-08T04:08:23.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.185+0000 7f12aa7fc640 1 -- 172.21.15.170:0/969768800 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c035020 con 0x7f12ac14b3b0 2024-05-08T04:08:23.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.185+0000 7f427b59e640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f427c072a60 0x7f427c142ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.202 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.185+0000 7f12aa7fc640 1 -- 172.21.15.170:0/969768800 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f129c031810 con 0x7f12ac14b3b0 2024-05-08T04:08:23.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.184+0000 7f7d75916640 1 -- 172.21.15.170:0/323260053 >> v2:172.21.15.170:3300/0 conn(0x7f7d680e2e50 msgr2=0x7f7d680e5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:23.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.184+0000 7f7d75916640 1 --2- 172.21.15.170:0/323260053 >> v2:172.21.15.170:3300/0 conn(0x7f7d680e2e50 0x7f7d680e5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.184+0000 7f7d75916640 1 -- 172.21.15.170:0/323260053 >> v1:172.21.15.170:6789/0 conn(0x7f7d68032e80 legacy=0x7f7d6802de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:23.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.184+0000 7f7d75916640 1 -- 172.21.15.170:0/323260053 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6802d9e0 con 0x7f7d680e5ec0 2024-05-08T04:08:23.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.185+0000 7f427a59c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f427c0700a0 0x7f427c070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.203 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.185+0000 7f427a59c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f427c0700a0 0x7f427c070de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:54418/0 (socket says 172.21.15.170:54418) 2024-05-08T04:08:23.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.185+0000 7f427a59c640 1 -- 172.21.15.170:0/2601876094 learned_addr learned my addr 172.21.15.170:0/2601876094 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:23.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.185+0000 7f7c4db17640 1 --2- 172.21.15.170:0/1512319638 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7c480712b0 0x7f7c48073ad0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000b080 tx=0x7f7c4002e2f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa74a69ff6fec649 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:23.204 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.185+0000 7f427b59e640 1 -- 172.21.15.170:0/2601876094 --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f427c072080 con 0x7f427c072ea0 2024-05-08T04:08:23.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.185+0000 7f427b59e640 1 -- 172.21.15.170:0/2601876094 --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7f427c071320 con 0x7f427c072a60 2024-05-08T04:08:23.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.185+0000 7f427b59e640 1 -- 172.21.15.170:0/2601876094 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f427c072360 con 0x7f427c0700a0 2024-05-08T04:08:23.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.185+0000 7f12aa7fc640 1 -- 172.21.15.170:0/969768800 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c031be0 con 0x7f12ac14b3b0 2024-05-08T04:08:23.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.185+0000 7f4daaffd640 1 -- 172.21.15.170:0/3053786822 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da4034440 con 0x7f4dac068c80 2024-05-08T04:08:23.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f4db22db640 1 -- 172.21.15.170:0/3053786822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4dac068c80 msgr2=0x7f4dac1470d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.205 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f4db22db640 1 --2- 172.21.15.170:0/3053786822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4dac068c80 0x7f4dac1470d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4dac06e8a0 tx=0x7f4da402bd90 comp rx=0 tx=0).stop 2024-05-08T04:08:23.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.185+0000 7f8ba5c45640 1 -- 172.21.15.170:0/376684602 shutdown_connections 2024-05-08T04:08:23.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.185+0000 7f8ba5c45640 1 --2- 172.21.15.170:0/376684602 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8b900051c0 0x7f8b9000b9c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.185+0000 7f8ba5c45640 1 -- 172.21.15.170:0/376684602 >> 172.21.15.170:0/376684602 conn(0x7f8b980061f0 msgr2=0x7f8b98006600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:23.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.185+0000 7f8ba5c45640 1 -- 172.21.15.170:0/376684602 shutdown_connections 2024-05-08T04:08:23.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f12b1fce640 1 -- 172.21.15.170:0/969768800 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f12ac14b3b0 msgr2=0x7f12ac14b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.206 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f12b1fce640 1 --2- 172.21.15.170:0/969768800 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f12ac14b3b0 0x7f12ac14b800 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f129c0097a0 tx=0x7f129c02b8b0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f7d75916640 1 --2- 172.21.15.170:0/323260053 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7d680e5ec0 0x7f7d680ea0c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7d600097a0 tx=0x7f7d6002b780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ffa4a8ad27f0fa2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:23.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.185+0000 7f4279d9b640 1 -- 172.21.15.170:0/2601876094 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3963296163 0 0) 0x7f427c072080 con 0x7f427c072ea0 2024-05-08T04:08:23.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.185+0000 7f4279d9b640 1 -- 172.21.15.170:0/2601876094 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4268003520 con 0x7f427c072ea0 2024-05-08T04:08:23.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f4279d9b640 1 -- 172.21.15.170:0/2601876094 <== mon.1 v1:172.21.15.187:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3440461795 0 0) 0x7f4268003520 con 0x7f427c072ea0 2024-05-08T04:08:23.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f4273fff640 1 --2- 172.21.15.170:0/2601876094 >> v2:172.21.15.170:3300/0 conn(0x7f427c072a60 0x7f427c142ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f4279d9b640 1 -- 172.21.15.170:0/2601876094 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f427c0700a0 msgr2=0x7f427c070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:23.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f4279d9b640 1 --2- 172.21.15.170:0/2601876094 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f427c0700a0 0x7f427c070de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f4279d9b640 1 -- 172.21.15.170:0/2601876094 >> v2:172.21.15.170:3300/0 conn(0x7f427c072a60 msgr2=0x7f427c142ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:23.207 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f4279d9b640 1 --2- 172.21.15.170:0/2601876094 >> v2:172.21.15.170:3300/0 conn(0x7f427c072a60 0x7f427c142ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f4279d9b640 1 -- 172.21.15.170:0/2601876094 --> v1:172.21.15.187:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f427c072640 con 0x7f427c072ea0 2024-05-08T04:08:23.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f4279d9b640 1 -- 172.21.15.170:0/2601876094 <== mon.1 v1:172.21.15.187:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 1210551688 0 0) 0x7f4274001a70 con 0x7f427c072ea0 2024-05-08T04:08:23.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f4279d9b640 1 -- 172.21.15.170:0/2601876094 >> v1:172.21.15.187:6789/0 conn(0x7f427c072ea0 legacy=0x7f427c143220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f4279d9b640 1 --2- 172.21.15.170:0/2601876094 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4268003a00 0x7f4268003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f4279d9b640 1 --2- 172.21.15.170:0/2601876094 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f427c0700a0 0x7f4268004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f4279d9b640 1 --2- 172.21.15.170:0/2601876094 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4268004d40 0x7f42680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f4279d9b640 1 -- 172.21.15.170:0/2601876094 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f427c071320 con 0x7f4268003a00 2024-05-08T04:08:23.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f4279d9b640 1 -- 172.21.15.170:0/2601876094 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f427c072360 con 0x7f4268004d40 2024-05-08T04:08:23.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f4279d9b640 1 -- 172.21.15.170:0/2601876094 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f426800bc50 con 0x7f427c0700a0 2024-05-08T04:08:23.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f4273fff640 1 --2- 172.21.15.170:0/2601876094 >> v2:172.21.15.170:3300/0 conn(0x7f427c072a60 0x7f427c142ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:08:23.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f427ad9d640 1 --2- 172.21.15.170:0/2601876094 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4268004d40 0x7f42680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.208 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f427a59c640 1 --2- 172.21.15.170:0/2601876094 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4268003a00 0x7f4268003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.185+0000 7f6ec363a640 1 Processor -- start 2024-05-08T04:08:23.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f6ec363a640 1 -- start start 2024-05-08T04:08:23.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.185+0000 7f7c4d316640 1 -- 172.21.15.170:0/1512319638 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4003b070 con 0x7f7c480712b0 2024-05-08T04:08:23.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.185+0000 7f7c4d316640 1 -- 172.21.15.170:0/1512319638 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c40036440 con 0x7f7c480712b0 2024-05-08T04:08:23.209 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f4273fff640 1 --2- 172.21.15.170:0/2601876094 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f427c0700a0 0x7f4268004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f7c4d316640 1 -- 172.21.15.170:0/1512319638 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c40036810 con 0x7f7c480712b0 2024-05-08T04:08:23.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f7c4f31a640 1 -- 172.21.15.170:0/1512319638 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7c480712b0 msgr2=0x7f7c48073ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f7c4f31a640 1 --2- 172.21.15.170:0/1512319638 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7c480712b0 0x7f7c48073ad0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000b080 tx=0x7f7c4002e2f0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f427ad9d640 1 -- 172.21.15.170:0/2601876094 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f427c0700a0 msgr2=0x7f4268004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:23.210 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f7c4f31a640 1 -- 172.21.15.170:0/1512319638 shutdown_connections 2024-05-08T04:08:23.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f12b1fce640 1 -- 172.21.15.170:0/969768800 shutdown_connections 2024-05-08T04:08:23.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f12b1fce640 1 --2- 172.21.15.170:0/969768800 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f12ac14b3b0 0x7f12ac14b800 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f12b1fce640 1 --2- 172.21.15.170:0/969768800 >> v2:172.21.15.187:3300/0 conn(0x7f12ac1454c0 0x7f12ac070500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f12b1fce640 1 -- 172.21.15.170:0/969768800 >> 172.21.15.170:0/969768800 conn(0x7f12ac0b4e30 msgr2=0x7f12ac0b5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:23.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f427ad9d640 1 --2- 172.21.15.170:0/2601876094 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f427c0700a0 0x7f4268004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f427ad9d640 1 -- 172.21.15.170:0/2601876094 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4268003a00 msgr2=0x7f4268003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:23.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f427ad9d640 1 --2- 172.21.15.170:0/2601876094 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4268003a00 0x7f4268003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f427ad9d640 1 -- 172.21.15.170:0/2601876094 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f427c072640 con 0x7f4268004d40 2024-05-08T04:08:23.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f4db22db640 1 -- 172.21.15.170:0/3053786822 shutdown_connections 2024-05-08T04:08:23.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f4db22db640 1 --2- 172.21.15.170:0/3053786822 >> v2:172.21.15.187:3300/0 conn(0x7f4dac0712c0 0x7f4dac0b1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f4db22db640 1 --2- 172.21.15.170:0/3053786822 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4dac147610 0x7f4dac070d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f4db22db640 1 --2- 172.21.15.170:0/3053786822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4dac068c80 0x7f4dac1470d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.187+0000 7f4db22db640 1 -- 172.21.15.170:0/3053786822 >> 172.21.15.170:0/3053786822 conn(0x7f4dac0b5c20 msgr2=0x7f4dac0b20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:23.212 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f8ba5c45640 1 -- 172.21.15.170:0/376684602 wait complete. 2024-05-08T04:08:23.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f7c4f31a640 1 --2- 172.21.15.170:0/1512319638 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7c480712b0 0x7f7c48073ad0 secure :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000b080 tx=0x7f7c4002e2f0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.186+0000 7f7c4f31a640 1 -- 172.21.15.170:0/1512319638 >> 172.21.15.170:0/1512319638 conn(0x7f7c480b5d40 msgr2=0x7f7c480b6150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:23.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.187+0000 7f12b1fce640 1 -- 172.21.15.170:0/969768800 shutdown_connections 2024-05-08T04:08:23.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.187+0000 7f12b1fce640 1 -- 172.21.15.170:0/969768800 wait complete. 2024-05-08T04:08:23.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.187+0000 7f6ec363a640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6eb402dcb0 0x7f6eb402efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.187+0000 7f6ec363a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6eb40c4450 0x7f6eb40c4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.187+0000 7f6ec363a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6eb402c4d0 con 0x7f6eb4033970 2024-05-08T04:08:23.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.187+0000 7f6ec363a640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7f6eb402cd50 con 0x7f6eb402dcb0 2024-05-08T04:08:23.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.187+0000 7f6ec363a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f6eb402fd70 con 0x7f6eb40c4450 2024-05-08T04:08:23.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.187+0000 7f6ec0bd5640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6eb402dcb0 0x7f6eb402efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.187+0000 7f6ec0bd5640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f6eb402dcb0 0x7f6eb402efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49164/0 (socket says 172.21.15.170:49164) 2024-05-08T04:08:23.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.187+0000 7f6ec0bd5640 1 -- 172.21.15.170:0/2783382881 learned_addr learned my addr 172.21.15.170:0/2783382881 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:23.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.187+0000 7f7c4f31a640 1 -- 172.21.15.170:0/1512319638 shutdown_connections 2024-05-08T04:08:23.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.187+0000 7f6ec1bd7640 1 --2- 172.21.15.170:0/2783382881 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6eb40c4450 0x7f6eb40c4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.187+0000 7f4db22db640 1 -- 172.21.15.170:0/3053786822 shutdown_connections 2024-05-08T04:08:23.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.188+0000 7f6ec0bd5640 1 -- 172.21.15.170:0/2783382881 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6eb40c4450 msgr2=0x7f6eb40c4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:23.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.187+0000 7f7d67fff640 1 -- 172.21.15.170:0/323260053 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d60037070 con 0x7f7d680e5ec0 2024-05-08T04:08:23.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.187+0000 7f7d67fff640 1 -- 172.21.15.170:0/323260053 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d60030710 con 0x7f7d680e5ec0 2024-05-08T04:08:23.215 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.187+0000 7f7d67fff640 1 -- 172.21.15.170:0/323260053 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d60030ae0 con 0x7f7d680e5ec0 2024-05-08T04:08:23.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.188+0000 7f4db22db640 1 -- 172.21.15.170:0/3053786822 wait complete. 2024-05-08T04:08:23.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.188+0000 7f12b1fce640 1 Processor -- start 2024-05-08T04:08:23.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.188+0000 7f427ad9d640 1 --2- 172.21.15.170:0/2601876094 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4268004d40 0x7f42680051c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f427400e6e0 tx=0x7f4268005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c32626dc6eed2841 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:23.216 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.188+0000 7f12b1fce640 1 -- start start 2024-05-08T04:08:23.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.188+0000 7f4279d9b640 1 -- 172.21.15.170:0/2601876094 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427403a070 con 0x7f4268004d40 2024-05-08T04:08:23.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.188+0000 7f4279d9b640 1 -- 172.21.15.170:0/2601876094 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4274035620 con 0x7f4268004d40 2024-05-08T04:08:23.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.188+0000 7f4279d9b640 1 -- 172.21.15.170:0/2601876094 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427400e330 con 0x7f4268004d40 2024-05-08T04:08:23.217 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.188+0000 7f7d76117640 1 -- 172.21.15.170:0/323260053 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7d680e5ec0 msgr2=0x7f7d680ea0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.187+0000 7f7c4f31a640 1 -- 172.21.15.170:0/1512319638 wait complete. 2024-05-08T04:08:23.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.188+0000 7f7c4f31a640 1 Processor -- start 2024-05-08T04:08:23.218 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.188+0000 7f7d76117640 1 --2- 172.21.15.170:0/323260053 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7d680e5ec0 0x7f7d680ea0c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7d600097a0 tx=0x7f7d6002b780 comp rx=0 tx=0).stop 2024-05-08T04:08:23.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.189+0000 7f7d76117640 1 -- 172.21.15.170:0/323260053 shutdown_connections 2024-05-08T04:08:23.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.189+0000 7f7d76117640 1 --2- 172.21.15.170:0/323260053 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7d680e5ec0 0x7f7d680ea0c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.189+0000 7f7d76117640 1 --2- 172.21.15.170:0/323260053 >> v2:172.21.15.170:3300/0 conn(0x7f7d680e2e50 0x7f7d680e5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.189+0000 7f7d76117640 1 -- 172.21.15.170:0/323260053 >> 172.21.15.170:0/323260053 conn(0x7f7d68036620 msgr2=0x7f7d68005e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:23.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.188+0000 7f8ba5c45640 1 Processor -- start 2024-05-08T04:08:23.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.188+0000 7f8ba5c45640 1 -- start start 2024-05-08T04:08:23.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.188+0000 7f6ec0bd5640 1 --2- 172.21.15.170:0/2783382881 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6eb40c4450 0x7f6eb40c4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.219 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.188+0000 7f6ec0bd5640 1 -- 172.21.15.170:0/2783382881 >> v1:172.21.15.187:6789/0 conn(0x7f6eb4033970 legacy=0x7f6eb40342d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.188+0000 7f6ec0bd5640 1 -- 172.21.15.170:0/2783382881 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb402d1e0 con 0x7f6eb402dcb0 2024-05-08T04:08:23.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.189+0000 7f6ec0bd5640 1 --2- 172.21.15.170:0/2783382881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6eb402dcb0 0x7f6eb402efd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8009870 tx=0x7f6eb802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f74bfbbb605f9a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:23.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.189+0000 7f7d76117640 1 -- 172.21.15.170:0/323260053 shutdown_connections 2024-05-08T04:08:23.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.189+0000 7f12b1fce640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f12ac070120 0x7f12ac1e9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.220 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.189+0000 7f12b1fce640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12ac1454c0 0x7f12ac1e9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.189+0000 7f7d76117640 1 -- 172.21.15.170:0/323260053 wait complete. 2024-05-08T04:08:23.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.189+0000 7f6eb3fff640 1 -- 172.21.15.170:0/2783382881 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8034070 con 0x7f6eb402dcb0 2024-05-08T04:08:23.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.189+0000 7f6eb3fff640 1 -- 172.21.15.170:0/2783382881 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6eb8030860 con 0x7f6eb402dcb0 2024-05-08T04:08:23.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.189+0000 7f427b59e640 1 -- 172.21.15.170:0/2601876094 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4268004d40 msgr2=0x7f42680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.221 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.189+0000 7f427b59e640 1 --2- 172.21.15.170:0/2601876094 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4268004d40 0x7f42680051c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f427400e6e0 tx=0x7f4268005830 comp rx=0 tx=0).stop 2024-05-08T04:08:23.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.189+0000 7f4db22db640 1 Processor -- start 2024-05-08T04:08:23.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.189+0000 7f12b1fce640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12ac14b3b0 0x7f12ac11f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.190+0000 7f427ad9d640 1 -- 172.21.15.170:0/2601876094 reap_dead start 2024-05-08T04:08:23.222 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.189+0000 7f7c4f31a640 1 -- start start 2024-05-08T04:08:23.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.190+0000 7f427b59e640 1 -- 172.21.15.170:0/2601876094 shutdown_connections 2024-05-08T04:08:23.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.190+0000 7f427b59e640 1 -- 172.21.15.170:0/2601876094 >> 172.21.15.170:0/2601876094 conn(0x7f427c0b4bb0 msgr2=0x7f427c0b4f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:23.223 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.190+0000 7f12b1fce640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac152460 con 0x7f12ac14b3b0 2024-05-08T04:08:23.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.190+0000 7f12b1fce640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac14bf50 con 0x7f12ac1454c0 2024-05-08T04:08:23.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.190+0000 7f12b1fce640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f12ac1525e0 con 0x7f12ac070120 2024-05-08T04:08:23.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.190+0000 7f12abfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12ac14b3b0 0x7f12ac11f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.190+0000 7f4db22db640 1 -- start start 2024-05-08T04:08:23.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.190+0000 7f6eb3fff640 1 -- 172.21.15.170:0/2783382881 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8030c70 con 0x7f6eb402dcb0 2024-05-08T04:08:23.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.190+0000 7f12abfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12ac14b3b0 0x7f12ac11f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49174/0 (socket says 172.21.15.170:49174) 2024-05-08T04:08:23.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.190+0000 7f427b59e640 1 -- 172.21.15.170:0/2601876094 shutdown_connections 2024-05-08T04:08:23.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.190+0000 7f12abfff640 1 -- 172.21.15.170:0/2244246697 learned_addr learned my addr 172.21.15.170:0/2244246697 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:23.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.190+0000 7f12aaffd640 1 --2- 172.21.15.170:0/2244246697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12ac1454c0 0x7f12ac1e9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.225 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.190+0000 7f427b59e640 1 -- 172.21.15.170:0/2601876094 wait complete. 2024-05-08T04:08:23.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.191+0000 7f6ec363a640 1 -- 172.21.15.170:0/2783382881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6eb402dcb0 msgr2=0x7f6eb402efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.191+0000 7f6ec363a640 1 --2- 172.21.15.170:0/2783382881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6eb402dcb0 0x7f6eb402efd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8009870 tx=0x7f6eb802b9a0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.191+0000 7f12abfff640 1 -- 172.21.15.170:0/2244246697 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f12ac070120 msgr2=0x7f12ac1e9400 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T04:08:23.226 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.191+0000 7f7d76117640 1 Processor -- start 2024-05-08T04:08:23.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.191+0000 7f6ec363a640 1 -- 172.21.15.170:0/2783382881 shutdown_connections 2024-05-08T04:08:23.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.191+0000 7f6ec363a640 1 --2- 172.21.15.170:0/2783382881 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6eb40c4450 0x7f6eb40c4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.191+0000 7f6ec363a640 1 --2- 172.21.15.170:0/2783382881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6eb402dcb0 0x7f6eb402efd0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.191+0000 7f6ec363a640 1 -- 172.21.15.170:0/2783382881 >> 172.21.15.170:0/2783382881 conn(0x7f6eb4036610 msgr2=0x7f6eb4005e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:23.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.191+0000 7f6ec363a640 1 -- 172.21.15.170:0/2783382881 shutdown_connections 2024-05-08T04:08:23.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.191+0000 7f4db22db640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4dac068c80 0x7f4dac1e94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.190+0000 7f8ba5c45640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8b900051c0 0x7f8b9802f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.190+0000 7f8ba5c45640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b98030050 0x7f8b980a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.190+0000 7f8ba5c45640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b980a6810 0x7f8b980b7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.190+0000 7f8ba5c45640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8b980cfc00 con 0x7f8b980a6810 2024-05-08T04:08:23.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.190+0000 7f8ba5c45640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8b980334d0 con 0x7f8b98030050 2024-05-08T04:08:23.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.190+0000 7f8ba5c45640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f8b980cfd80 con 0x7f8b900051c0 2024-05-08T04:08:23.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.190+0000 7f8b9f7fe640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8b900051c0 0x7f8b9802f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.190+0000 7f8b9f7fe640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8b900051c0 0x7f8b9802f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:54454/0 (socket says 172.21.15.170:54454) 2024-05-08T04:08:23.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.190+0000 7f8b9f7fe640 1 -- 172.21.15.170:0/2656971298 learned_addr learned my addr 172.21.15.170:0/2656971298 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:23.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.190+0000 7f8b9effd640 1 --2- 172.21.15.170:0/2656971298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b98030050 0x7f8b980a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.228 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.191+0000 7f8b9effd640 1 -- 172.21.15.170:0/2656971298 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8b900051c0 msgr2=0x7f8b9802f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.191+0000 7f8b9effd640 1 --2- 172.21.15.170:0/2656971298 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8b900051c0 0x7f8b9802f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.191+0000 7f8b9effd640 1 -- 172.21.15.170:0/2656971298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b980a6810 msgr2=0x7f8b980b7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.191+0000 7f0af891c640 1 Processor -- start 2024-05-08T04:08:23.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.191+0000 7f0af891c640 1 -- start start 2024-05-08T04:08:23.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.191+0000 7f4db22db640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4dac0712c0 0x7f4dac1e99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.191+0000 7f8b9effd640 1 --2- 172.21.15.170:0/2656971298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b980a6810 0x7f8b980b7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.229 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.191+0000 7f8b9effd640 1 -- 172.21.15.170:0/2656971298 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b98031520 con 0x7f8b98030050 2024-05-08T04:08:23.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.191+0000 7f8b9effd640 1 --2- 172.21.15.170:0/2656971298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b98030050 0x7f8b980a62d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00b440 tx=0x7f8b8c00b900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:23.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.191+0000 7f12ab7fe640 1 --2- 172.21.15.170:0/2244246697 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f12ac070120 0x7f12ac1e9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.191+0000 7f6ec363a640 1 -- 172.21.15.170:0/2783382881 wait complete. 2024-05-08T04:08:23.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.189+0000 7f7c4f31a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7c48070ed0 0x7f7c481cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.189+0000 7f7c4f31a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c48074010 0x7f7c481c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.189+0000 7f7c4f31a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c481c9ea0 0x7f7c481e09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.189+0000 7f7c4f31a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7c481403b0 con 0x7f7c48074010 2024-05-08T04:08:23.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.189+0000 7f7c4f31a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7c48140230 con 0x7f7c481c9ea0 2024-05-08T04:08:23.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.189+0000 7f7c4f31a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f7c48075c30 con 0x7f7c48070ed0 2024-05-08T04:08:23.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.190+0000 7f7c4db17640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c48074010 0x7f7c481c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.190+0000 7f7c4db17640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c48074010 0x7f7c481c9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49168/0 (socket says 172.21.15.170:49168) 2024-05-08T04:08:23.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.190+0000 7f7c4db17640 1 -- 172.21.15.170:0/141841402 learned_addr learned my addr 172.21.15.170:0/141841402 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:23.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.191+0000 7f7c4db17640 1 -- 172.21.15.170:0/141841402 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7c48070ed0 msgr2=0x7f7c481cbdc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T04:08:23.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.191+0000 7f7c4db17640 1 --2- 172.21.15.170:0/141841402 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7c48070ed0 0x7f7c481cbdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.191+0000 7f7c4db17640 1 -- 172.21.15.170:0/141841402 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c481c9ea0 msgr2=0x7f7c481e09e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T04:08:23.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.191+0000 7f7c4db17640 1 --2- 172.21.15.170:0/141841402 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c481c9ea0 0x7f7c481e09e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.191+0000 7f7c4db17640 1 -- 172.21.15.170:0/141841402 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c48142310 con 0x7f7c48074010 2024-05-08T04:08:23.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.191+0000 7f7c4db17640 1 --2- 172.21.15.170:0/141841402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c48074010 0x7f7c481c9960 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7c4002e900 tx=0x7f7c40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:23.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.191+0000 7f4db22db640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4dac147610 0x7f4dac0aac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.191+0000 7f12abfff640 1 --2- 172.21.15.170:0/2244246697 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f12ac070120 0x7f12ac1e9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.191+0000 7f12abfff640 1 -- 172.21.15.170:0/2244246697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12ac1454c0 msgr2=0x7f12ac1e9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.191+0000 7f12abfff640 1 --2- 172.21.15.170:0/2244246697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12ac1454c0 0x7f12ac1e9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.191+0000 7f8b9f7fe640 1 --2- 172.21.15.170:0/2656971298 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8b900051c0 0x7f8b9802f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:08:23.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f4dabfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4dac068c80 0x7f4dac1e94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f12abfff640 1 -- 172.21.15.170:0/2244246697 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12ac11ef30 con 0x7f12ac14b3b0 2024-05-08T04:08:23.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.191+0000 7f7d76117640 1 -- start start 2024-05-08T04:08:23.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f7d76117640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d68032e80 0x7f7d680eeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f7d76117640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d680e2e50 0x7f7d680eb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f7d76117640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7d680e5ec0 0x7f7d680ebaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f4dab7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4dac0712c0 0x7f4dac1e99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f4dab7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4dac0712c0 0x7f4dac1e99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.170:47204/0 (socket says 172.21.15.170:47204) 2024-05-08T04:08:23.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f4dab7fe640 1 -- 172.21.15.170:0/2586270471 learned_addr learned my addr 172.21.15.170:0/2586270471 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:23.234 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f427b59e640 1 Processor -- start 2024-05-08T04:08:23.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f4db0878640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4dac147610 0x7f4dac0aac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f4db22db640 1 -- 172.21.15.170:0/2586270471 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4dac070370 con 0x7f4dac068c80 2024-05-08T04:08:23.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f4db22db640 1 -- 172.21.15.170:0/2586270471 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4dac071720 con 0x7f4dac0712c0 2024-05-08T04:08:23.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f8b8bfff640 1 -- 172.21.15.170:0/2656971298 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c015020 con 0x7f8b98030050 2024-05-08T04:08:23.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f8b8bfff640 1 -- 172.21.15.170:0/2656971298 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b8c011830 con 0x7f8b98030050 2024-05-08T04:08:23.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f427b59e640 1 -- start start 2024-05-08T04:08:23.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f4db22db640 1 -- 172.21.15.170:0/2586270471 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f4dac0704f0 con 0x7f4dac147610 2024-05-08T04:08:23.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f8b8bfff640 1 -- 172.21.15.170:0/2656971298 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c011c00 con 0x7f8b98030050 2024-05-08T04:08:23.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f8ba5c45640 1 -- 172.21.15.170:0/2656971298 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b98031f00 con 0x7f8b98030050 2024-05-08T04:08:23.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f8ba5c45640 1 -- 172.21.15.170:0/2656971298 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b980cf930 con 0x7f8b98030050 2024-05-08T04:08:23.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f4dab7fe640 1 -- 172.21.15.170:0/2586270471 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4dac147610 msgr2=0x7f4dac0aac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.193+0000 7f4dab7fe640 1 --2- 172.21.15.170:0/2586270471 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4dac147610 0x7f4dac0aac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.193+0000 7f4dab7fe640 1 -- 172.21.15.170:0/2586270471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4dac068c80 msgr2=0x7f4dac1e94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.193+0000 7f4dab7fe640 1 --2- 172.21.15.170:0/2586270471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4dac068c80 0x7f4dac1e94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.193+0000 7f4dab7fe640 1 -- 172.21.15.170:0/2586270471 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dac0aa690 con 0x7f4dac0712c0 2024-05-08T04:08:23.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f12aaffd640 1 --2- 172.21.15.170:0/2244246697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12ac1454c0 0x7f12ac1e9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:08:23.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f12ab7fe640 1 --2- 172.21.15.170:0/2244246697 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f12ac070120 0x7f12ac1e9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:08:23.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f7d75115640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d68032e80 0x7f7d680eeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f7d75115640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d68032e80 0x7f7d680eeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49198/0 (socket says 172.21.15.170:49198) 2024-05-08T04:08:23.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f7d76117640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7d680e3440 con 0x7f7d68032e80 2024-05-08T04:08:23.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f7d76117640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7d680e32c0 con 0x7f7d680e2e50 2024-05-08T04:08:23.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f7d76117640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f7d680e6320 con 0x7f7d680e5ec0 2024-05-08T04:08:23.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f7d75115640 1 -- 172.21.15.170:0/1586314429 learned_addr learned my addr 172.21.15.170:0/1586314429 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:23.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f7d74914640 1 --2- 172.21.15.170:0/1586314429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d680e2e50 0x7f7d680eb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f0af891c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0af40740c0 0x7f0af4140820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f0af2ffd640 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f0af41403b0 0x7f0af40739b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:55984/0 (socket says 172.21.15.170:55984) 2024-05-08T04:08:23.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f0af37fe640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f0af4071560 0x7f0af40af690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.170:50376/0 (socket says 172.21.15.170:50376) 2024-05-08T04:08:23.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f0af891c640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0af4073040 con 0x7f0af41403b0 2024-05-08T04:08:23.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f0af891c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0af40732b0 con 0x7f0af4071560 2024-05-08T04:08:23.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f0af891c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f0af4070df0 con 0x7f0af40740c0 2024-05-08T04:08:23.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f0af3fff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0af40740c0 0x7f0af4140820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f0af3fff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0af40740c0 0x7f0af4140820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:54476/0 (socket says 172.21.15.170:54476) 2024-05-08T04:08:23.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.192+0000 7f0af3fff640 1 -- 172.21.15.170:0/1341576250 learned_addr learned my addr 172.21.15.170:0/1341576250 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:23.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.193+0000 7f12abfff640 1 --2- 172.21.15.170:0/2244246697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12ac14b3b0 0x7f12ac11f6f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f129c034040 tx=0x7f129c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:23.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.193+0000 7f7c3effd640 1 -- 172.21.15.170:0/141841402 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c40044070 con 0x7f7c48074010 2024-05-08T04:08:23.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.193+0000 7f7c3effd640 1 -- 172.21.15.170:0/141841402 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c40003b90 con 0x7f7c48074010 2024-05-08T04:08:23.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.193+0000 7f427b59e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4268004d40 0x7f427c0b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.193+0000 7f427b59e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f427c13caa0 0x7f427c13e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.193+0000 7f427b59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f427c13ba10 0x7f427c0abda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.193+0000 7f7d74914640 1 -- 172.21.15.170:0/1586314429 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7d680e5ec0 msgr2=0x7f7d680ebaa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T04:08:23.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.193+0000 7f7d74914640 1 --2- 172.21.15.170:0/1586314429 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7d680e5ec0 0x7f7d680ebaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.193+0000 7f427b59e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f427c070600 con 0x7f427c13ba10 2024-05-08T04:08:23.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.193+0000 7f427b59e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f427c070480 con 0x7f427c13caa0 2024-05-08T04:08:23.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.193+0000 7f427b59e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f427c143930 con 0x7f4268004d40 2024-05-08T04:08:23.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.193+0000 7f7d74914640 1 -- 172.21.15.170:0/1586314429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d68032e80 msgr2=0x7f7d680eeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.193+0000 7f7d74914640 1 --2- 172.21.15.170:0/1586314429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d68032e80 0x7f7d680eeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.193+0000 7f7d74914640 1 -- 172.21.15.170:0/1586314429 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d680eab80 con 0x7f7d680e2e50 2024-05-08T04:08:23.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.193+0000 7f6ec363a640 1 Processor -- start 2024-05-08T04:08:23.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.193+0000 7f6ec363a640 1 -- start start 2024-05-08T04:08:23.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.193+0000 7f7d74914640 1 --2- 172.21.15.170:0/1586314429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d680e2e50 0x7f7d680eb560 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7d700520f0 tx=0x7f7d70069700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:23.241 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.193+0000 7f7c3effd640 1 -- 172.21.15.170:0/141841402 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c4003b040 con 0x7f7c48074010 2024-05-08T04:08:23.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.193+0000 7f4dab7fe640 1 --2- 172.21.15.170:0/2586270471 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4dac0712c0 0x7f4dac1e99f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c00e870 tx=0x7f4d9c00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:23.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.193+0000 7f7d65ffb640 1 -- 172.21.15.170:0/1586314429 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d70074620 con 0x7f7d680e2e50 2024-05-08T04:08:23.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.193+0000 7f7d65ffb640 1 -- 172.21.15.170:0/1586314429 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d700747c0 con 0x7f7d680e2e50 2024-05-08T04:08:23.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.194+0000 7f7d65ffb640 1 -- 172.21.15.170:0/1586314429 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7006d450 con 0x7f7d680e2e50 2024-05-08T04:08:23.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.194+0000 7f7d76117640 1 -- 172.21.15.170:0/1586314429 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d680ea0c0 con 0x7f7d680e2e50 2024-05-08T04:08:23.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.193+0000 7f0af27fc640 1 -- 172.21.15.170:0/1341576250 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3297607350 0 0) 0x7f0af40732b0 con 0x7f0af4071560 2024-05-08T04:08:23.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.193+0000 7f0af27fc640 1 -- 172.21.15.170:0/1341576250 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ad8003520 con 0x7f0af4071560 2024-05-08T04:08:23.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.194+0000 7f427ad9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f427c13ba10 0x7f427c0abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.194+0000 7f427ad9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f427c13ba10 0x7f427c0abda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49206/0 (socket says 172.21.15.170:49206) 2024-05-08T04:08:23.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.194+0000 7f427ad9d640 1 -- 172.21.15.170:0/1589030644 learned_addr learned my addr 172.21.15.170:0/1589030644 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:23.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.194+0000 7f7d76117640 1 -- 172.21.15.170:0/1586314429 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d680eb140 con 0x7f7d680e2e50 2024-05-08T04:08:23.243 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.194+0000 7f6ec363a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6eb402dcb0 0x7f6eb4029120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.194+0000 7f427a59c640 1 --2- 172.21.15.170:0/1589030644 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4268004d40 0x7f427c0b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.194+0000 7f427ad9d640 1 -- 172.21.15.170:0/1589030644 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4268004d40 msgr2=0x7f427c0b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.194+0000 7f6ec363a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6eb4033970 0x7f6eb40b0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.244 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.194+0000 7f6ec363a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6eb40c4450 0x7f6eb40b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.194+0000 7f6ec13d6640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6eb402dcb0 0x7f6eb4029120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.194+0000 7f6ec13d6640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6eb402dcb0 0x7f6eb4029120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:54504/0 (socket says 172.21.15.170:54504) 2024-05-08T04:08:23.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.195+0000 7f427ad9d640 1 --2- 172.21.15.170:0/1589030644 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4268004d40 0x7f427c0b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.195+0000 7f427ad9d640 1 -- 172.21.15.170:0/1589030644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f427c13caa0 msgr2=0x7f427c13e250 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T04:08:23.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.195+0000 7f427ad9d640 1 --2- 172.21.15.170:0/1589030644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f427c13caa0 0x7f427c13e250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.195+0000 7f427ad9d640 1 -- 172.21.15.170:0/1589030644 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f427c13dbc0 con 0x7f427c13ba10 2024-05-08T04:08:23.245 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.194+0000 7f6ec0bd5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6eb4033970 0x7f6eb40b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.194+0000 7f4da8ff9640 1 -- 172.21.15.170:0/2586270471 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c017070 con 0x7f4dac0712c0 2024-05-08T04:08:23.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.194+0000 7f6ec13d6640 1 -- 172.21.15.170:0/169509877 learned_addr learned my addr 172.21.15.170:0/169509877 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:23.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.194+0000 7f6ec363a640 1 -- 172.21.15.170:0/169509877 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb402d740 con 0x7f6eb40c4450 2024-05-08T04:08:23.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.194+0000 7f6ec363a640 1 -- 172.21.15.170:0/169509877 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb402d5c0 con 0x7f6eb4033970 2024-05-08T04:08:23.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.194+0000 7f6ec363a640 1 -- 172.21.15.170:0/169509877 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f6eb402d8c0 con 0x7f6eb402dcb0 2024-05-08T04:08:23.246 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.195+0000 7f427a59c640 1 --2- 172.21.15.170:0/1589030644 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4268004d40 0x7f427c0b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:08:23.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.194+0000 7f6ec1bd7640 1 --2- 172.21.15.170:0/169509877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6eb40c4450 0x7f6eb40b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.194+0000 7f4da8ff9640 1 -- 172.21.15.170:0/2586270471 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d9c00c940 con 0x7f4dac0712c0 2024-05-08T04:08:23.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.195+0000 7f6ec0bd5640 1 -- 172.21.15.170:0/169509877 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6eb402dcb0 msgr2=0x7f6eb4029120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.195+0000 7f6ec0bd5640 1 --2- 172.21.15.170:0/169509877 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6eb402dcb0 0x7f6eb4029120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.195+0000 7f6ec0bd5640 1 -- 172.21.15.170:0/169509877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6eb40c4450 msgr2=0x7f6eb40b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.194+0000 7f4da8ff9640 1 -- 172.21.15.170:0/2586270471 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c00cd30 con 0x7f4dac0712c0 2024-05-08T04:08:23.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.195+0000 7f4db22db640 1 -- 172.21.15.170:0/2586270471 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dac1362d0 con 0x7f4dac0712c0 2024-05-08T04:08:23.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.195+0000 7f4db22db640 1 -- 172.21.15.170:0/2586270471 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dac1e9f30 con 0x7f4dac0712c0 2024-05-08T04:08:23.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.195+0000 7f6ec0bd5640 1 --2- 172.21.15.170:0/169509877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6eb40c4450 0x7f6eb40b1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.195+0000 7f7d537fe640 1 -- 172.21.15.170:0/1586314429 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d68005390 con 0x7f7d680e2e50 2024-05-08T04:08:23.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.195+0000 7f6ec0bd5640 1 -- 172.21.15.170:0/169509877 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb40b12f0 con 0x7f6eb4033970 2024-05-08T04:08:23.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.195+0000 7f6ec1bd7640 1 --2- 172.21.15.170:0/169509877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6eb40c4450 0x7f6eb40b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:08:23.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.195+0000 7f427ad9d640 1 --2- 172.21.15.170:0/1589030644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f427c13ba10 0x7f427c0abda0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f427402ed60 tx=0x7f4274001c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:23.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.196+0000 7f6ec13d6640 1 --2- 172.21.15.170:0/169509877 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6eb402dcb0 0x7f6eb4029120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:08:23.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.194+0000 7f0af3fff640 1 -- 172.21.15.170:0/1341576250 >> v1:172.21.15.187:6789/0 conn(0x7f0af4071560 legacy=0x7f0af40af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.194+0000 7f0af3fff640 1 -- 172.21.15.170:0/1341576250 >> v1:172.21.15.170:6789/0 conn(0x7f0af41403b0 legacy=0x7f0af40739b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.194+0000 7f0af3fff640 1 -- 172.21.15.170:0/1341576250 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af4073590 con 0x7f0af40740c0 2024-05-08T04:08:23.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.194+0000 7f7c4f31a640 1 -- 172.21.15.170:0/141841402 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c481425f0 con 0x7f7c48074010 2024-05-08T04:08:23.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.194+0000 7f7c4f31a640 1 -- 172.21.15.170:0/141841402 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c481ca330 con 0x7f7c48074010 2024-05-08T04:08:23.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.196+0000 7f7c4f31a640 1 -- 172.21.15.170:0/141841402 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c481cb770 con 0x7f7c48074010 2024-05-08T04:08:23.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.194+0000 7f8ba5c45640 1 -- 172.21.15.170:0/2656971298 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b80005080 con 0x7f8b98030050 2024-05-08T04:08:23.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.196+0000 7f6ec0bd5640 1 --2- 172.21.15.170:0/169509877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6eb4033970 0x7f6eb40b0090 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8033000 tx=0x7f6eb8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:23.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.195+0000 7f8b8bfff640 1 -- 172.21.15.170:0/2656971298 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f8b8c018020 con 0x7f8b98030050 2024-05-08T04:08:23.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.196+0000 7f8b8bfff640 1 --2- 172.21.15.170:0/2656971298 >> v2:172.21.15.187:6816/101007897 conn(0x7f8b84076360 0x7f8b84078820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.196+0000 7f8b9f7fe640 1 --2- 172.21.15.170:0/2656971298 >> v2:172.21.15.187:6816/101007897 conn(0x7f8b84076360 0x7f8b84078820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.196+0000 7f8b8bfff640 1 -- 172.21.15.170:0/2656971298 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4090+0+0 (secure 0 0 0) 0x7f8b8c096d20 con 0x7f8b98030050 2024-05-08T04:08:23.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.196+0000 7f7d65ffb640 1 -- 172.21.15.170:0/1586314429 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f7d7006d5f0 con 0x7f7d680e2e50 2024-05-08T04:08:23.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.196+0000 7f1293fff640 1 -- 172.21.15.170:0/2244246697 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c040070 con 0x7f12ac14b3b0 2024-05-08T04:08:23.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.196+0000 7f1293fff640 1 -- 172.21.15.170:0/2244246697 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f129c039bf0 con 0x7f12ac14b3b0 2024-05-08T04:08:23.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.196+0000 7f0af3fff640 1 --2- 172.21.15.170:0/1341576250 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0af40740c0 0x7f0af4140820 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0aec009e80 tx=0x7f0aec02bd00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0fa07f1da5a88cd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:23.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.196+0000 7f1293fff640 1 -- 172.21.15.170:0/2244246697 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c040070 con 0x7f12ac14b3b0 2024-05-08T04:08:23.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.197+0000 7f8b9f7fe640 1 --2- 172.21.15.170:0/2656971298 >> v2:172.21.15.187:6816/101007897 conn(0x7f8b84076360 0x7f8b84078820 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8b94005880 tx=0x7f8b94009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:23.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.197+0000 7f12b1fce640 1 -- 172.21.15.170:0/2244246697 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12ac0b4450 con 0x7f12ac14b3b0 2024-05-08T04:08:23.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.197+0000 7f12b1fce640 1 -- 172.21.15.170:0/2244246697 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12ac1e9e80 con 0x7f12ac14b3b0 2024-05-08T04:08:23.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.197+0000 7f0af27fc640 1 -- 172.21.15.170:0/1341576250 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4001620 con 0x7f0af40740c0 2024-05-08T04:08:23.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.197+0000 7f0af27fc640 1 -- 172.21.15.170:0/1341576250 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0aec02c620 con 0x7f0af40740c0 2024-05-08T04:08:23.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.197+0000 7f0af27fc640 1 -- 172.21.15.170:0/1341576250 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec02c9d0 con 0x7f0af40740c0 2024-05-08T04:08:23.253 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.197+0000 7f4db22db640 1 -- 172.21.15.170:0/2586270471 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dac069bd0 con 0x7f4dac0712c0 2024-05-08T04:08:23.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.197+0000 7f4da8ff9640 1 -- 172.21.15.170:0/2586270471 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f4d9c01a070 con 0x7f4dac0712c0 2024-05-08T04:08:23.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.197+0000 7f7d65ffb640 1 --2- 172.21.15.170:0/1586314429 >> v2:172.21.15.187:6816/101007897 conn(0x7f7d54076360 0x7f7d54078820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.197+0000 7f7d65ffb640 1 -- 172.21.15.170:0/1586314429 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4090+0+0 (secure 0 0 0) 0x7f7d70101e80 con 0x7f7d680e2e50 2024-05-08T04:08:23.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.198+0000 7f7d75115640 1 --2- 172.21.15.170:0/1586314429 >> v2:172.21.15.187:6816/101007897 conn(0x7f7d54076360 0x7f7d54078820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.198+0000 7f4da8ff9640 1 --2- 172.21.15.170:0/2586270471 >> v2:172.21.15.187:6816/101007897 conn(0x7f4d980762e0 0x7f4d980787a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.198+0000 7f0af891c640 1 -- 172.21.15.170:0/1341576250 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0af40740c0 msgr2=0x7f0af4140820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.198+0000 7f0af891c640 1 --2- 172.21.15.170:0/1341576250 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0af40740c0 0x7f0af4140820 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0aec009e80 tx=0x7f0aec02bd00 comp rx=0 tx=0).stop 2024-05-08T04:08:23.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.198+0000 7f4dabfff640 1 --2- 172.21.15.170:0/2586270471 >> v2:172.21.15.187:6816/101007897 conn(0x7f4d980762e0 0x7f4d980787a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.198+0000 7f0af891c640 1 -- 172.21.15.170:0/1341576250 shutdown_connections 2024-05-08T04:08:23.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.198+0000 7f0af891c640 1 --2- 172.21.15.170:0/1341576250 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0af40740c0 0x7f0af4140820 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.198+0000 7f0af891c640 1 -- 172.21.15.170:0/1341576250 >> 172.21.15.170:0/1341576250 conn(0x7f0af40b5dd0 msgr2=0x7f0af40b61e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:23.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.198+0000 7f0af891c640 1 -- 172.21.15.170:0/1341576250 shutdown_connections 2024-05-08T04:08:23.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.198+0000 7f4272ffd640 1 -- 172.21.15.170:0/1589030644 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427403a040 con 0x7f427c13ba10 2024-05-08T04:08:23.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.198+0000 7f12b1fce640 1 -- 172.21.15.170:0/2244246697 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12ac14aef0 con 0x7f12ac14b3b0 2024-05-08T04:08:23.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.198+0000 7f7d75115640 1 --2- 172.21.15.170:0/1586314429 >> v2:172.21.15.187:6816/101007897 conn(0x7f7d54076360 0x7f7d54078820 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c0099a0 tx=0x7f7d6c0090e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:23.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.198+0000 7f4272ffd640 1 -- 172.21.15.170:0/1589030644 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42740359d0 con 0x7f427c13ba10 2024-05-08T04:08:23.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.198+0000 7f4272ffd640 1 -- 172.21.15.170:0/1589030644 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427400e330 con 0x7f427c13ba10 2024-05-08T04:08:23.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.198+0000 7f427b59e640 1 -- 172.21.15.170:0/1589030644 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f427c0adb10 con 0x7f427c13ba10 2024-05-08T04:08:23.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.198+0000 7f427b59e640 1 -- 172.21.15.170:0/1589030644 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f427c13be50 con 0x7f427c13ba10 2024-05-08T04:08:23.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.198+0000 7f0af891c640 1 -- 172.21.15.170:0/1341576250 wait complete. 2024-05-08T04:08:23.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.198+0000 7f4dabfff640 1 --2- 172.21.15.170:0/2586270471 >> v2:172.21.15.187:6816/101007897 conn(0x7f4d980762e0 0x7f4d980787a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4da4036000 tx=0x7f4da4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:23.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.198+0000 7f7c3effd640 1 -- 172.21.15.170:0/141841402 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f7c40047070 con 0x7f7c48074010 2024-05-08T04:08:23.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.199+0000 7f7c3effd640 1 --2- 172.21.15.170:0/141841402 >> v2:172.21.15.187:6816/101007897 conn(0x7f7c2c0762e0 0x7f7c2c0787a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.199+0000 7f1293fff640 1 -- 172.21.15.170:0/2244246697 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f129c04b060 con 0x7f12ac14b3b0 2024-05-08T04:08:23.258 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.199+0000 7f7c3effd640 1 -- 172.21.15.170:0/141841402 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4090+0+0 (secure 0 0 0) 0x7f7c400b8b20 con 0x7f7c48074010 2024-05-08T04:08:23.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.199+0000 7f427b59e640 1 -- 172.21.15.170:0/1589030644 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4260005080 con 0x7f427c13ba10 2024-05-08T04:08:23.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.200+0000 7f0af891c640 1 Processor -- start 2024-05-08T04:08:23.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.200+0000 7f7c4e318640 1 --2- 172.21.15.170:0/141841402 >> v2:172.21.15.187:6816/101007897 conn(0x7f7c2c0762e0 0x7f7c2c0787a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.200+0000 7f1293fff640 1 --2- 172.21.15.170:0/2244246697 >> v2:172.21.15.187:6816/101007897 conn(0x7f1298076490 0x7f1298078950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.200+0000 7f0af891c640 1 -- start start 2024-05-08T04:08:23.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.200+0000 7f12ab7fe640 1 --2- 172.21.15.170:0/2244246697 >> v2:172.21.15.187:6816/101007897 conn(0x7f1298076490 0x7f1298078950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.200+0000 7f7c4e318640 1 --2- 172.21.15.170:0/141841402 >> v2:172.21.15.187:6816/101007897 conn(0x7f7c2c0762e0 0x7f7c2c0787a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7c44003c50 tx=0x7f7c440079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:23.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.200+0000 7f12ab7fe640 1 --2- 172.21.15.170:0/2244246697 >> v2:172.21.15.187:6816/101007897 conn(0x7f1298076490 0x7f1298078950 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f12a0001220 tx=0x7f12a0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:23.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.200+0000 7f0af891c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0af4071560 0x7f0af414f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.200+0000 7f0af891c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0af40740c0 0x7f0af414cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.200+0000 7f0af891c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0af41403b0 0x7f0af414d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.200+0000 7f0af891c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0af4144c90 con 0x7f0af41403b0 2024-05-08T04:08:23.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.201+0000 7f0af891c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0af40afda0 con 0x7f0af4071560 2024-05-08T04:08:23.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.201+0000 7f0af891c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f0af4144e10 con 0x7f0af40740c0 2024-05-08T04:08:23.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.201+0000 7f0af2ffd640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0af40740c0 0x7f0af414cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.201+0000 7f0af37fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0af4071560 0x7f0af414f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.201+0000 7f0af3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0af41403b0 0x7f0af414d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.201+0000 7f0af2ffd640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0af40740c0 0x7f0af414cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:54506/0 (socket says 172.21.15.170:54506) 2024-05-08T04:08:23.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.201+0000 7f0af2ffd640 1 -- 172.21.15.170:0/2355408412 learned_addr learned my addr 172.21.15.170:0/2355408412 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:23.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.201+0000 7f4da8ff9640 1 -- 172.21.15.170:0/2586270471 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4090+0+0 (secure 0 0 0) 0x7f4d9c05cd40 con 0x7f4dac0712c0 2024-05-08T04:08:23.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.201+0000 7f0af2ffd640 1 -- 172.21.15.170:0/2355408412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0af4071560 msgr2=0x7f0af414f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.201+0000 7f0af2ffd640 1 --2- 172.21.15.170:0/2355408412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0af4071560 0x7f0af414f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.201+0000 7f0af2ffd640 1 -- 172.21.15.170:0/2355408412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0af41403b0 msgr2=0x7f0af414d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.201+0000 7f0af2ffd640 1 --2- 172.21.15.170:0/2355408412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0af41403b0 0x7f0af414d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.201+0000 7f0af2ffd640 1 -- 172.21.15.170:0/2355408412 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af40aa630 con 0x7f0af40740c0 2024-05-08T04:08:23.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.201+0000 7f0af37fe640 1 --2- 172.21.15.170:0/2355408412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0af4071560 0x7f0af414f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:08:23.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.201+0000 7f0af3fff640 1 --2- 172.21.15.170:0/2355408412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0af41403b0 0x7f0af414d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:08:23.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.201+0000 7f8b8bfff640 1 -- 172.21.15.170:0/2656971298 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8b8c01f020 con 0x7f8b98030050 2024-05-08T04:08:23.264 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.201+0000 7f0af2ffd640 1 --2- 172.21.15.170:0/2355408412 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0af40740c0 0x7f0af414cf40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0ae000c520 tx=0x7f0ae000c9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:23.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.201+0000 7f0aebfff640 1 -- 172.21.15.170:0/2355408412 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae0018070 con 0x7f0af40740c0 2024-05-08T04:08:23.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.201+0000 7f0aebfff640 1 -- 172.21.15.170:0/2355408412 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ae0007480 con 0x7f0af40740c0 2024-05-08T04:08:23.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.201+0000 7f0aebfff640 1 -- 172.21.15.170:0/2355408412 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae00047e0 con 0x7f0af40740c0 2024-05-08T04:08:23.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.201+0000 7f0af891c640 1 -- 172.21.15.170:0/2355408412 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0af40aa910 con 0x7f0af40740c0 2024-05-08T04:08:23.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.202+0000 7f0af891c640 1 -- 172.21.15.170:0/2355408412 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0af4150ec0 con 0x7f0af40740c0 2024-05-08T04:08:23.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.203+0000 7f0af891c640 1 -- 172.21.15.170:0/2355408412 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0af4070670 con 0x7f0af40740c0 2024-05-08T04:08:23.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.203+0000 7f0aebfff640 1 -- 172.21.15.170:0/2355408412 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f0ae0004980 con 0x7f0af40740c0 2024-05-08T04:08:23.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.204+0000 7f0aebfff640 1 --2- 172.21.15.170:0/2355408412 >> v2:172.21.15.187:6816/101007897 conn(0x7f0adc076290 0x7f0adc078750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.204+0000 7f0aebfff640 1 -- 172.21.15.170:0/2355408412 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 4090+0+0 (secure 0 0 0) 0x7f0ae00961f0 con 0x7f0af40740c0 2024-05-08T04:08:23.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.204+0000 7f0af37fe640 1 --2- 172.21.15.170:0/2355408412 >> v2:172.21.15.187:6816/101007897 conn(0x7f0adc076290 0x7f0adc078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.204+0000 7f0af37fe640 1 --2- 172.21.15.170:0/2355408412 >> v2:172.21.15.187:6816/101007897 conn(0x7f0adc076290 0x7f0adc078750 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4001eb0 tx=0x7f0ae40019e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:23.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.206+0000 7f6eb1ffb640 1 -- 172.21.15.170:0/169509877 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8048070 con 0x7f6eb4033970 2024-05-08T04:08:23.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.206+0000 7f6ec363a640 1 -- 172.21.15.170:0/169509877 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eb40b1560 con 0x7f6eb4033970 2024-05-08T04:08:23.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.206+0000 7f7d65ffb640 1 -- 172.21.15.170:0/1586314429 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7d700cbea0 con 0x7f7d680e2e50 2024-05-08T04:08:23.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.206+0000 7f6ec363a640 1 -- 172.21.15.170:0/169509877 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eb402b590 con 0x7f6eb4033970 2024-05-08T04:08:23.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.206+0000 7f7c3effd640 1 -- 172.21.15.170:0/141841402 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7c4004e020 con 0x7f7c48074010 2024-05-08T04:08:23.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.207+0000 7f1293fff640 1 -- 172.21.15.170:0/2244246697 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4090+0+0 (secure 0 0 0) 0x7f129c07e450 con 0x7f12ac14b3b0 2024-05-08T04:08:23.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.207+0000 7f6ec363a640 1 -- 172.21.15.170:0/169509877 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eb4005380 con 0x7f6eb4033970 2024-05-08T04:08:23.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.208+0000 7f4272ffd640 1 -- 172.21.15.170:0/1589030644 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f4274005070 con 0x7f427c13ba10 2024-05-08T04:08:23.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.209+0000 7f4272ffd640 1 --2- 172.21.15.170:0/1589030644 >> v2:172.21.15.187:6816/101007897 conn(0x7f425c076310 0x7f425c0787d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.209+0000 7f427a59c640 1 --2- 172.21.15.170:0/1589030644 >> v2:172.21.15.187:6816/101007897 conn(0x7f425c076310 0x7f425c0787d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.209+0000 7f427a59c640 1 --2- 172.21.15.170:0/1589030644 >> v2:172.21.15.187:6816/101007897 conn(0x7f425c076310 0x7f425c0787d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f426c006cc0 tx=0x7f426c00c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:23.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.210+0000 7f6eb1ffb640 1 -- 172.21.15.170:0/169509877 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6eb80094b0 con 0x7f6eb4033970 2024-05-08T04:08:23.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.210+0000 7f6eb1ffb640 1 -- 172.21.15.170:0/169509877 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8052060 con 0x7f6eb4033970 2024-05-08T04:08:23.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.210+0000 7f4da8ff9640 1 -- 172.21.15.170:0/2586270471 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4dac1362d0 con 0x7f4dac0712c0 2024-05-08T04:08:23.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.213+0000 7f0aebfff640 1 -- 172.21.15.170:0/2355408412 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0af4070670 con 0x7f0af40740c0 2024-05-08T04:08:23.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.214+0000 7f6eb1ffb640 1 -- 172.21.15.170:0/169509877 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f6eb804b070 con 0x7f6eb4033970 2024-05-08T04:08:23.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.215+0000 7f6eb1ffb640 1 --2- 172.21.15.170:0/169509877 >> v2:172.21.15.187:6816/101007897 conn(0x7f6ea0076290 0x7f6ea0078750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:23.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.215+0000 7f6eb1ffb640 1 -- 172.21.15.170:0/169509877 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4090+0+0 (secure 0 0 0) 0x7f6eb80b92e0 con 0x7f6eb4033970 2024-05-08T04:08:23.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.215+0000 7f6ec13d6640 1 --2- 172.21.15.170:0/169509877 >> v2:172.21.15.187:6816/101007897 conn(0x7f6ea0076290 0x7f6ea0078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:23.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.215+0000 7f6eb1ffb640 1 -- 172.21.15.170:0/169509877 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6eb8082cb0 con 0x7f6eb4033970 2024-05-08T04:08:23.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.215+0000 7f6ec13d6640 1 --2- 172.21.15.170:0/169509877 >> v2:172.21.15.187:6816/101007897 conn(0x7f6ea0076290 0x7f6ea0078750 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0512d0 tx=0x7f6ebc06bdc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:23.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.217+0000 7f1293fff640 1 -- 172.21.15.170:0/2244246697 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f12ac14aef0 con 0x7f12ac14b3b0 2024-05-08T04:08:23.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.225+0000 7f4272ffd640 1 -- 172.21.15.170:0/1589030644 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4090+0+0 (secure 0 0 0) 0x7f427407ec80 con 0x7f427c13ba10 2024-05-08T04:08:23.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.226+0000 7f4272ffd640 1 -- 172.21.15.170:0/1589030644 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4274082690 con 0x7f427c13ba10 2024-05-08T04:08:23.638 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.636+0000 7f8ba5c45640 1 -- 172.21.15.170:0/2656971298 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f8b80005370 con 0x7f8b98030050 2024-05-08T04:08:23.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.637+0000 7f8b8bfff640 1 -- 172.21.15.170:0/2656971298 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8b8c060cb0 con 0x7f8b98030050 2024-05-08T04:08:23.639 INFO:teuthology.orchestra.run.smithi170.stdout:60129542146 2024-05-08T04:08:23.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.638+0000 7f7c4f31a640 1 -- 172.21.15.170:0/141841402 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f7c480507b0 con 0x7f7c48074010 2024-05-08T04:08:23.641 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.639+0000 7f7c3effd640 1 -- 172.21.15.170:0/141841402 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7c40082c40 con 0x7f7c48074010 2024-05-08T04:08:23.641 INFO:teuthology.orchestra.run.smithi170.stdout:60129542146 2024-05-08T04:08:23.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.641+0000 7f8b89ffb640 1 -- 172.21.15.170:0/2656971298 >> v2:172.21.15.187:6816/101007897 conn(0x7f8b84076360 msgr2=0x7f8b84078820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.641+0000 7f8b89ffb640 1 --2- 172.21.15.170:0/2656971298 >> v2:172.21.15.187:6816/101007897 conn(0x7f8b84076360 0x7f8b84078820 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8b94005880 tx=0x7f8b94009040 comp rx=0 tx=0).stop 2024-05-08T04:08:23.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.641+0000 7f8b89ffb640 1 -- 172.21.15.170:0/2656971298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b98030050 msgr2=0x7f8b980a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.641+0000 7f8b89ffb640 1 --2- 172.21.15.170:0/2656971298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b98030050 0x7f8b980a62d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00b440 tx=0x7f8b8c00b900 comp rx=0 tx=0).stop 2024-05-08T04:08:23.643 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.641+0000 7f8b89ffb640 1 -- 172.21.15.170:0/2656971298 shutdown_connections 2024-05-08T04:08:23.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.641+0000 7f8b89ffb640 1 --2- 172.21.15.170:0/2656971298 >> v2:172.21.15.187:6816/101007897 conn(0x7f8b84076360 0x7f8b84078820 secure :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8b94005880 tx=0x7f8b94009040 comp rx=0 tx=0).stop 2024-05-08T04:08:23.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.641+0000 7f8b89ffb640 1 --2- 172.21.15.170:0/2656971298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8b98030050 0x7f8b980a62d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.641+0000 7f8b89ffb640 1 --2- 172.21.15.170:0/2656971298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b980a6810 0x7f8b980b7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.641+0000 7f8b89ffb640 1 --2- 172.21.15.170:0/2656971298 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8b900051c0 0x7f8b9802f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.641+0000 7f8b89ffb640 1 -- 172.21.15.170:0/2656971298 >> 172.21.15.170:0/2656971298 conn(0x7f8b980061f0 msgr2=0x7f8b980c0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:23.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.642+0000 7f8b89ffb640 1 -- 172.21.15.170:0/2656971298 shutdown_connections 2024-05-08T04:08:23.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.642+0000 7f8b89ffb640 1 -- 172.21.15.170:0/2656971298 wait complete. 2024-05-08T04:08:23.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.642+0000 7f7c4f31a640 1 -- 172.21.15.170:0/141841402 >> v2:172.21.15.187:6816/101007897 conn(0x7f7c2c0762e0 msgr2=0x7f7c2c0787a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.644 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.642+0000 7f7c4f31a640 1 --2- 172.21.15.170:0/141841402 >> v2:172.21.15.187:6816/101007897 conn(0x7f7c2c0762e0 0x7f7c2c0787a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7c44003c50 tx=0x7f7c440079b0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.642+0000 7f7c4f31a640 1 -- 172.21.15.170:0/141841402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c48074010 msgr2=0x7f7c481c9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.642+0000 7f7c4f31a640 1 --2- 172.21.15.170:0/141841402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c48074010 0x7f7c481c9960 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7c4002e900 tx=0x7f7c40004a40 comp rx=0 tx=0).stop 2024-05-08T04:08:23.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.642+0000 7f7d537fe640 1 -- 172.21.15.170:0/1586314429 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f7d680ecb60 con 0x7f7d680e2e50 2024-05-08T04:08:23.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.643+0000 7f7c4f31a640 1 -- 172.21.15.170:0/141841402 shutdown_connections 2024-05-08T04:08:23.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.643+0000 7f7c4f31a640 1 --2- 172.21.15.170:0/141841402 >> v2:172.21.15.187:6816/101007897 conn(0x7f7c2c0762e0 0x7f7c2c0787a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.643+0000 7f7c4f31a640 1 --2- 172.21.15.170:0/141841402 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7c481c9ea0 0x7f7c481e09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.643+0000 7f7c4f31a640 1 --2- 172.21.15.170:0/141841402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c48074010 0x7f7c481c9960 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.643+0000 7f7c4f31a640 1 --2- 172.21.15.170:0/141841402 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7c48070ed0 0x7f7c481cbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.645 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.643+0000 7f7c4f31a640 1 -- 172.21.15.170:0/141841402 >> 172.21.15.170:0/141841402 conn(0x7f7c480b5d40 msgr2=0x7f7c4813c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:23.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.643+0000 7f7c4f31a640 1 -- 172.21.15.170:0/141841402 shutdown_connections 2024-05-08T04:08:23.646 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.643+0000 7f7d65ffb640 1 -- 172.21.15.170:0/1586314429 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7d700cb7a0 con 0x7f7d680e2e50 2024-05-08T04:08:23.646 INFO:teuthology.orchestra.run.smithi170.stdout:60129542146 2024-05-08T04:08:23.647 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.643+0000 7f7c4f31a640 1 -- 172.21.15.170:0/141841402 wait complete. 2024-05-08T04:08:23.649 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.647+0000 7f7d76117640 1 -- 172.21.15.170:0/1586314429 >> v2:172.21.15.187:6816/101007897 conn(0x7f7d54076360 msgr2=0x7f7d54078820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.647+0000 7f7d76117640 1 --2- 172.21.15.170:0/1586314429 >> v2:172.21.15.187:6816/101007897 conn(0x7f7d54076360 0x7f7d54078820 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c0099a0 tx=0x7f7d6c0090e0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.647+0000 7f7d76117640 1 -- 172.21.15.170:0/1586314429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d680e2e50 msgr2=0x7f7d680eb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.647+0000 7f7d76117640 1 --2- 172.21.15.170:0/1586314429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d680e2e50 0x7f7d680eb560 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7d700520f0 tx=0x7f7d70069700 comp rx=0 tx=0).stop 2024-05-08T04:08:23.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.647+0000 7f7d76117640 1 -- 172.21.15.170:0/1586314429 shutdown_connections 2024-05-08T04:08:23.650 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.647+0000 7f7d76117640 1 --2- 172.21.15.170:0/1586314429 >> v2:172.21.15.187:6816/101007897 conn(0x7f7d54076360 0x7f7d54078820 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.647+0000 7f7d76117640 1 --2- 172.21.15.170:0/1586314429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7d680e2e50 0x7f7d680eb560 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.647+0000 7f7d76117640 1 --2- 172.21.15.170:0/1586314429 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7d680e5ec0 0x7f7d680ebaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.647+0000 7f7d76117640 1 --2- 172.21.15.170:0/1586314429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d68032e80 0x7f7d680eeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.647+0000 7f7d76117640 1 -- 172.21.15.170:0/1586314429 >> 172.21.15.170:0/1586314429 conn(0x7f7d68036620 msgr2=0x7f7d680e2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:23.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.647+0000 7f7d76117640 1 -- 172.21.15.170:0/1586314429 shutdown_connections 2024-05-08T04:08:23.651 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.647+0000 7f7d76117640 1 -- 172.21.15.170:0/1586314429 wait complete. 2024-05-08T04:08:23.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.650+0000 7f4db22db640 1 -- 172.21.15.170:0/2586270471 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f4dac145f30 con 0x7f4dac0712c0 2024-05-08T04:08:23.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.650+0000 7f0af891c640 1 -- 172.21.15.170:0/2355408412 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f0af4072730 con 0x7f0af40740c0 2024-05-08T04:08:23.652 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.650+0000 7f4da8ff9640 1 -- 172.21.15.170:0/2586270471 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4d9c060750 con 0x7f4dac0712c0 2024-05-08T04:08:23.653 INFO:teuthology.orchestra.run.smithi170.stdout:60129542146 2024-05-08T04:08:23.653 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.652+0000 7f6ec363a640 1 -- 172.21.15.170:0/169509877 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f6eb4034790 con 0x7f6eb4033970 2024-05-08T04:08:23.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.652+0000 7f6eb1ffb640 1 -- 172.21.15.170:0/169509877 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6eb8086fb0 con 0x7f6eb4033970 2024-05-08T04:08:23.654 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.652+0000 7f12b1fce640 1 -- 172.21.15.170:0/2244246697 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f12ac06b400 con 0x7f12ac14b3b0 2024-05-08T04:08:23.655 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.652+0000 7f0aebfff640 1 -- 172.21.15.170:0/2355408412 <== mon.2 v2:172.21.15.170:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0af4072730 con 0x7f0af40740c0 2024-05-08T04:08:23.655 INFO:teuthology.orchestra.run.smithi170.stdout:60129542146 2024-05-08T04:08:23.655 INFO:teuthology.orchestra.run.smithi170.stdout:55834574850 2024-05-08T04:08:23.656 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.653+0000 7f1293fff640 1 -- 172.21.15.170:0/2244246697 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f12ac14aef0 con 0x7f12ac14b3b0 2024-05-08T04:08:23.656 INFO:teuthology.orchestra.run.smithi170.stdout:60129542146 2024-05-08T04:08:23.657 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.654+0000 7f4db22db640 1 -- 172.21.15.170:0/2586270471 >> v2:172.21.15.187:6816/101007897 conn(0x7f4d980762e0 msgr2=0x7f4d980787a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.654+0000 7f4db22db640 1 --2- 172.21.15.170:0/2586270471 >> v2:172.21.15.187:6816/101007897 conn(0x7f4d980762e0 0x7f4d980787a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4da4036000 tx=0x7f4da4030000 comp rx=0 tx=0).stop 2024-05-08T04:08:23.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.654+0000 7f4db22db640 1 -- 172.21.15.170:0/2586270471 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4dac0712c0 msgr2=0x7f4dac1e99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.654+0000 7f4db22db640 1 --2- 172.21.15.170:0/2586270471 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4dac0712c0 0x7f4dac1e99f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c00e870 tx=0x7f4d9c00ed30 comp rx=0 tx=0).stop 2024-05-08T04:08:23.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.654+0000 7f4db22db640 1 -- 172.21.15.170:0/2586270471 shutdown_connections 2024-05-08T04:08:23.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.654+0000 7f4db22db640 1 --2- 172.21.15.170:0/2586270471 >> v2:172.21.15.187:6816/101007897 conn(0x7f4d980762e0 0x7f4d980787a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.655+0000 7f4db22db640 1 --2- 172.21.15.170:0/2586270471 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4dac0712c0 0x7f4dac1e99f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.655+0000 7f4db22db640 1 --2- 172.21.15.170:0/2586270471 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4dac147610 0x7f4dac0aac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.655+0000 7f4db22db640 1 --2- 172.21.15.170:0/2586270471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4dac068c80 0x7f4dac1e94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.655+0000 7f4db22db640 1 -- 172.21.15.170:0/2586270471 >> 172.21.15.170:0/2586270471 conn(0x7f4dac0b5c20 msgr2=0x7f4dac068180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:23.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.655+0000 7f4db22db640 1 -- 172.21.15.170:0/2586270471 shutdown_connections 2024-05-08T04:08:23.658 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.655+0000 7f4db22db640 1 -- 172.21.15.170:0/2586270471 wait complete. 2024-05-08T04:08:23.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.656+0000 7f6ec363a640 1 -- 172.21.15.170:0/169509877 >> v2:172.21.15.187:6816/101007897 conn(0x7f6ea0076290 msgr2=0x7f6ea0078750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.657+0000 7f6ec363a640 1 --2- 172.21.15.170:0/169509877 >> v2:172.21.15.187:6816/101007897 conn(0x7f6ea0076290 0x7f6ea0078750 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0512d0 tx=0x7f6ebc06bdc0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.657+0000 7f6ec363a640 1 -- 172.21.15.170:0/169509877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6eb4033970 msgr2=0x7f6eb40b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.657+0000 7f6ec363a640 1 --2- 172.21.15.170:0/169509877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6eb4033970 0x7f6eb40b0090 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8033000 tx=0x7f6eb8004730 comp rx=0 tx=0).stop 2024-05-08T04:08:23.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.657+0000 7f6ec363a640 1 -- 172.21.15.170:0/169509877 shutdown_connections 2024-05-08T04:08:23.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.657+0000 7f6ec363a640 1 --2- 172.21.15.170:0/169509877 >> v2:172.21.15.187:6816/101007897 conn(0x7f6ea0076290 0x7f6ea0078750 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.657+0000 7f6ec363a640 1 --2- 172.21.15.170:0/169509877 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6eb4033970 0x7f6eb40b0090 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.657+0000 7f6ec363a640 1 --2- 172.21.15.170:0/169509877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6eb40c4450 0x7f6eb40b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.657+0000 7f6ec363a640 1 --2- 172.21.15.170:0/169509877 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6eb402dcb0 0x7f6eb4029120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.660 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.657+0000 7f6ec363a640 1 -- 172.21.15.170:0/169509877 >> 172.21.15.170:0/169509877 conn(0x7f6eb4036610 msgr2=0x7f6eb4030f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:23.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.657+0000 7f6ec363a640 1 -- 172.21.15.170:0/169509877 shutdown_connections 2024-05-08T04:08:23.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.657+0000 7f6ec363a640 1 -- 172.21.15.170:0/169509877 wait complete. 2024-05-08T04:08:23.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.657+0000 7f0ae9ffb640 1 -- 172.21.15.170:0/2355408412 >> v2:172.21.15.187:6816/101007897 conn(0x7f0adc076290 msgr2=0x7f0adc078750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.657+0000 7f0ae9ffb640 1 --2- 172.21.15.170:0/2355408412 >> v2:172.21.15.187:6816/101007897 conn(0x7f0adc076290 0x7f0adc078750 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4001eb0 tx=0x7f0ae40019e0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.657+0000 7f0ae9ffb640 1 -- 172.21.15.170:0/2355408412 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0af40740c0 msgr2=0x7f0af414cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.661 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.658+0000 7f0ae9ffb640 1 --2- 172.21.15.170:0/2355408412 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0af40740c0 0x7f0af414cf40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0ae000c520 tx=0x7f0ae000c9e0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.658+0000 7f12b1fce640 1 -- 172.21.15.170:0/2244246697 >> v2:172.21.15.187:6816/101007897 conn(0x7f1298076490 msgr2=0x7f1298078950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.662 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.658+0000 7f12b1fce640 1 --2- 172.21.15.170:0/2244246697 >> v2:172.21.15.187:6816/101007897 conn(0x7f1298076490 0x7f1298078950 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f12a0001220 tx=0x7f12a0009040 comp rx=0 tx=0).stop 2024-05-08T04:08:23.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.658+0000 7f0ae9ffb640 1 -- 172.21.15.170:0/2355408412 shutdown_connections 2024-05-08T04:08:23.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.658+0000 7f0ae9ffb640 1 --2- 172.21.15.170:0/2355408412 >> v2:172.21.15.187:6816/101007897 conn(0x7f0adc076290 0x7f0adc078750 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.658+0000 7f0ae9ffb640 1 --2- 172.21.15.170:0/2355408412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0af41403b0 0x7f0af414d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.658+0000 7f0ae9ffb640 1 --2- 172.21.15.170:0/2355408412 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0af40740c0 0x7f0af414cf40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.658+0000 7f0ae9ffb640 1 --2- 172.21.15.170:0/2355408412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0af4071560 0x7f0af414f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.658+0000 7f0ae9ffb640 1 -- 172.21.15.170:0/2355408412 >> 172.21.15.170:0/2355408412 conn(0x7f0af40b5dd0 msgr2=0x7f0af4073950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:23.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.658+0000 7f12b1fce640 1 -- 172.21.15.170:0/2244246697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12ac14b3b0 msgr2=0x7f12ac11f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.663 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.658+0000 7f12b1fce640 1 --2- 172.21.15.170:0/2244246697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12ac14b3b0 0x7f12ac11f6f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f129c034040 tx=0x7f129c0046c0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.658+0000 7f0ae9ffb640 1 -- 172.21.15.170:0/2355408412 shutdown_connections 2024-05-08T04:08:23.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.658+0000 7f12b1fce640 1 -- 172.21.15.170:0/2244246697 shutdown_connections 2024-05-08T04:08:23.664 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.658+0000 7f12b1fce640 1 --2- 172.21.15.170:0/2244246697 >> v2:172.21.15.187:6816/101007897 conn(0x7f1298076490 0x7f1298078950 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.658+0000 7f0ae9ffb640 1 -- 172.21.15.170:0/2355408412 wait complete. 2024-05-08T04:08:23.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.658+0000 7f12b1fce640 1 --2- 172.21.15.170:0/2244246697 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12ac1454c0 0x7f12ac1e9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.659+0000 7f12b1fce640 1 --2- 172.21.15.170:0/2244246697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12ac14b3b0 0x7f12ac11f6f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.665 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.659+0000 7f12b1fce640 1 --2- 172.21.15.170:0/2244246697 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f12ac070120 0x7f12ac1e9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.659+0000 7f12b1fce640 1 -- 172.21.15.170:0/2244246697 >> 172.21.15.170:0/2244246697 conn(0x7f12ac0b4e30 msgr2=0x7f12ac144b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:23.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.659+0000 7f12b1fce640 1 -- 172.21.15.170:0/2244246697 shutdown_connections 2024-05-08T04:08:23.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.659+0000 7f12b1fce640 1 -- 172.21.15.170:0/2244246697 wait complete. 2024-05-08T04:08:23.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.661+0000 7f427b59e640 1 -- 172.21.15.170:0/1589030644 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f4260005640 con 0x7f427c13ba10 2024-05-08T04:08:23.666 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.662+0000 7f4272ffd640 1 -- 172.21.15.170:0/1589030644 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4274086990 con 0x7f427c13ba10 2024-05-08T04:08:23.666 INFO:teuthology.orchestra.run.smithi170.stdout:60129542146 2024-05-08T04:08:23.668 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542146 for osd.7 2024-05-08T04:08:23.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.666+0000 7f427b59e640 1 -- 172.21.15.170:0/1589030644 >> v2:172.21.15.187:6816/101007897 conn(0x7f425c076310 msgr2=0x7f425c0787d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.666+0000 7f427b59e640 1 --2- 172.21.15.170:0/1589030644 >> v2:172.21.15.187:6816/101007897 conn(0x7f425c076310 0x7f425c0787d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f426c006cc0 tx=0x7f426c00c040 comp rx=0 tx=0).stop 2024-05-08T04:08:23.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.666+0000 7f427b59e640 1 -- 172.21.15.170:0/1589030644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f427c13ba10 msgr2=0x7f427c0abda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:23.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.666+0000 7f427b59e640 1 --2- 172.21.15.170:0/1589030644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f427c13ba10 0x7f427c0abda0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f427402ed60 tx=0x7f4274001c10 comp rx=0 tx=0).stop 2024-05-08T04:08:23.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.667+0000 7f427b59e640 1 -- 172.21.15.170:0/1589030644 shutdown_connections 2024-05-08T04:08:23.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.667+0000 7f427b59e640 1 --2- 172.21.15.170:0/1589030644 >> v2:172.21.15.187:6816/101007897 conn(0x7f425c076310 0x7f425c0787d0 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f426c006cc0 tx=0x7f426c00c040 comp rx=0 tx=0).stop 2024-05-08T04:08:23.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.667+0000 7f427b59e640 1 --2- 172.21.15.170:0/1589030644 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f427c13caa0 0x7f427c13e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.667+0000 7f427b59e640 1 --2- 172.21.15.170:0/1589030644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f427c13ba10 0x7f427c0abda0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.667+0000 7f427b59e640 1 --2- 172.21.15.170:0/1589030644 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4268004d40 0x7f427c0b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:23.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.667+0000 7f427b59e640 1 -- 172.21.15.170:0/1589030644 >> 172.21.15.170:0/1589030644 conn(0x7f427c0b4bb0 msgr2=0x7f427c1c92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:23.669 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.667+0000 7f427b59e640 1 -- 172.21.15.170:0/1589030644 shutdown_connections 2024-05-08T04:08:23.670 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:23.667+0000 7f427b59e640 1 -- 172.21.15.170:0/1589030644 wait complete. 2024-05-08T04:08:23.670 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542146 for osd.0 2024-05-08T04:08:23.674 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542146 for osd.4 2024-05-08T04:08:23.680 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542146 for osd.6 2024-05-08T04:08:23.681 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542146 for osd.5 2024-05-08T04:08:23.683 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542146 for osd.1 2024-05-08T04:08:23.684 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574850 for osd.2 2024-05-08T04:08:23.685 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542146 for osd.3 2024-05-08T04:08:24.668 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-08T04:08:24.671 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T04:08:24.674 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T04:08:24.681 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-08T04:08:24.682 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T04:08:24.684 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T04:08:24.684 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T04:08:24.686 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T04:08:24.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.832+0000 7f5353fff640 1 Processor -- start 2024-05-08T04:08:24.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.832+0000 7f5353fff640 1 -- start start 2024-05-08T04:08:24.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.833+0000 7f5353fff640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f535406fc80 0x7f53540af970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.834+0000 7f5353fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f535414bd60 0x7f5354152450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.834+0000 7f5353fff640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5354071910 con 0x7f53540739c0 2024-05-08T04:08:24.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.834+0000 7f5353fff640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7f5354070060 con 0x7f535406fc80 2024-05-08T04:08:24.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.834+0000 7f5353fff640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5354071bf0 con 0x7f535414bd60 2024-05-08T04:08:24.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.835+0000 7f5352ffd640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f535406fc80 0x7f53540af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:24.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.835+0000 7f5352ffd640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f535406fc80 0x7f53540af970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49228/0 (socket says 172.21.15.170:49228) 2024-05-08T04:08:24.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.835+0000 7f5352ffd640 1 -- 172.21.15.170:0/3807655415 learned_addr learned my addr 172.21.15.170:0/3807655415 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:24.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.835+0000 7f5352ffd640 1 -- 172.21.15.170:0/3807655415 >> v2:172.21.15.187:3300/0 conn(0x7f535414bd60 msgr2=0x7f5354152450 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T04:08:24.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.835+0000 7f5352ffd640 1 --2- 172.21.15.170:0/3807655415 >> v2:172.21.15.187:3300/0 conn(0x7f535414bd60 0x7f5354152450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.838 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.835+0000 7f5352ffd640 1 -- 172.21.15.170:0/3807655415 >> v1:172.21.15.170:6789/0 conn(0x7f53540739c0 legacy=0x7f535414b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:24.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.835+0000 7f5352ffd640 1 -- 172.21.15.170:0/3807655415 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5354071ed0 con 0x7f535406fc80 2024-05-08T04:08:24.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.836+0000 7f5352ffd640 1 --2- 172.21.15.170:0/3807655415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f535406fc80 0x7f53540af970 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f53540b1170 tx=0x7f534c02bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4686310f0e10bbf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:24.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.837+0000 7f5351ffb640 1 -- 172.21.15.170:0/3807655415 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c02caa0 con 0x7f535406fc80 2024-05-08T04:08:24.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.837+0000 7f5351ffb640 1 -- 172.21.15.170:0/3807655415 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f534c02cc40 con 0x7f535406fc80 2024-05-08T04:08:24.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.838+0000 7f5353fff640 1 -- 172.21.15.170:0/3807655415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f535406fc80 msgr2=0x7f53540af970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:24.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.838+0000 7f5353fff640 1 --2- 172.21.15.170:0/3807655415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f535406fc80 0x7f53540af970 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f53540b1170 tx=0x7f534c02bf00 comp rx=0 tx=0).stop 2024-05-08T04:08:24.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.838+0000 7f5353fff640 1 -- 172.21.15.170:0/3807655415 shutdown_connections 2024-05-08T04:08:24.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.838+0000 7f5353fff640 1 --2- 172.21.15.170:0/3807655415 >> v2:172.21.15.187:3300/0 conn(0x7f535414bd60 0x7f5354152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.838+0000 7f5353fff640 1 --2- 172.21.15.170:0/3807655415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f535406fc80 0x7f53540af970 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.838+0000 7f5353fff640 1 -- 172.21.15.170:0/3807655415 >> 172.21.15.170:0/3807655415 conn(0x7f53540b4bb0 msgr2=0x7f53540b4f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:24.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.839+0000 7f5353fff640 1 -- 172.21.15.170:0/3807655415 shutdown_connections 2024-05-08T04:08:24.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.830+0000 7f1b7c816640 1 Processor -- start 2024-05-08T04:08:24.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.830+0000 7f1b7c816640 1 -- start start 2024-05-08T04:08:24.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.839+0000 7f5353fff640 1 -- 172.21.15.170:0/3807655415 wait complete. 2024-05-08T04:08:24.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.840+0000 7f5353fff640 1 Processor -- start 2024-05-08T04:08:24.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.841+0000 7f5353fff640 1 -- start start 2024-05-08T04:08:24.844 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.841+0000 7f5353fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f535406fc80 0x7f535407f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.845 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.841+0000 7f5353fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53540739c0 0x7f535407e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.841+0000 7f5353fff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f535414bd60 0x7f53541ee480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.841+0000 7f5353fff640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5354070310 con 0x7f53540739c0 2024-05-08T04:08:24.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.842+0000 7f5353fff640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5354070ab0 con 0x7f535406fc80 2024-05-08T04:08:24.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.842+0000 7f5353fff640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f53540afeb0 con 0x7f535414bd60 2024-05-08T04:08:24.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.842+0000 7f53537fe640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f535414bd60 0x7f53541ee480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:24.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.842+0000 7f53537fe640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f535414bd60 0x7f53541ee480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:54510/0 (socket says 172.21.15.170:54510) 2024-05-08T04:08:24.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.842+0000 7f53537fe640 1 -- 172.21.15.170:0/745347369 learned_addr learned my addr 172.21.15.170:0/745347369 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:24.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.842+0000 7f5352ffd640 1 --2- 172.21.15.170:0/745347369 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f535406fc80 0x7f535407f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:24.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.842+0000 7f5352ffd640 1 -- 172.21.15.170:0/745347369 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f535414bd60 msgr2=0x7f53541ee480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:24.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.842+0000 7f5352ffd640 1 --2- 172.21.15.170:0/745347369 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f535414bd60 0x7f53541ee480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.842+0000 7f5352ffd640 1 -- 172.21.15.170:0/745347369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53540739c0 msgr2=0x7f535407e480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T04:08:24.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.842+0000 7f5352ffd640 1 --2- 172.21.15.170:0/745347369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53540739c0 0x7f535407e480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.842+0000 7f5352ffd640 1 -- 172.21.15.170:0/745347369 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f535407f6d0 con 0x7f535406fc80 2024-05-08T04:08:24.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.843+0000 7f5352ffd640 1 --2- 172.21.15.170:0/745347369 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f535406fc80 0x7f535407f190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f53540b14f0 tx=0x7f534c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:24.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.844+0000 7f533b7fe640 1 -- 172.21.15.170:0/745347369 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c043030 con 0x7f535406fc80 2024-05-08T04:08:24.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.844+0000 7f5353fff640 1 -- 172.21.15.170:0/745347369 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f535407e9c0 con 0x7f535406fc80 2024-05-08T04:08:24.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.844+0000 7f5353fff640 1 -- 172.21.15.170:0/745347369 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f535407c1a0 con 0x7f535406fc80 2024-05-08T04:08:24.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.847+0000 7f533b7fe640 1 -- 172.21.15.170:0/745347369 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f534c03e450 con 0x7f535406fc80 2024-05-08T04:08:24.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.847+0000 7f533b7fe640 1 -- 172.21.15.170:0/745347369 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534c03e640 con 0x7f535406fc80 2024-05-08T04:08:24.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.848+0000 7f533b7fe640 1 -- 172.21.15.170:0/745347369 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f534c046070 con 0x7f535406fc80 2024-05-08T04:08:24.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.849+0000 7f533b7fe640 1 --2- 172.21.15.170:0/745347369 >> v2:172.21.15.187:6816/101007897 conn(0x7f5340076390 0x7f5340078850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.849+0000 7f533b7fe640 1 -- 172.21.15.170:0/745347369 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4090+0+0 (secure 0 0 0) 0x7f534c0b9630 con 0x7f535406fc80 2024-05-08T04:08:24.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.849+0000 7f53527fc640 1 --2- 172.21.15.170:0/745347369 >> v2:172.21.15.187:6816/101007897 conn(0x7f5340076390 0x7f5340078850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:24.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.831+0000 7f1b7c816640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1b70035240 0x7f1b700c16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.831+0000 7f1b7c816640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b7002d470 con 0x7f1b70035620 2024-05-08T04:08:24.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.831+0000 7f1b7c816640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b700c76d0 con 0x7f1b700c45b0 2024-05-08T04:08:24.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.831+0000 7f1b7c816640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f1b70032570 con 0x7f1b70035240 2024-05-08T04:08:24.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.831+0000 7f1b7759e640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f1b700c45b0 0x7f1b700c4a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.170:50390/0 (socket says 172.21.15.170:50390) 2024-05-08T04:08:24.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.850+0000 7f53527fc640 1 --2- 172.21.15.170:0/745347369 >> v2:172.21.15.187:6816/101007897 conn(0x7f5340076390 0x7f5340078850 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f53541f2420 tx=0x7f53480079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:24.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.832+0000 7f1b7759e640 1 -- 172.21.15.170:0/2018109361 learned_addr learned my addr 172.21.15.170:0/2018109361 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-05-08T04:08:24.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.850+0000 7f5353fff640 1 -- 172.21.15.170:0/745347369 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5334005080 con 0x7f535406fc80 2024-05-08T04:08:24.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.832+0000 7f1b75d9b640 1 -- 172.21.15.170:0/2018109361 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1480352268 0 0) 0x7f1b78051630 con 0x7f1b700c45b0 2024-05-08T04:08:24.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.832+0000 7f1b76d9d640 1 --2- 172.21.15.170:0/2018109361 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1b70035240 0x7f1b700c16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:24.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.832+0000 7f1b75d9b640 1 -- 172.21.15.170:0/2018109361 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b60003520 con 0x7f1b700c45b0 2024-05-08T04:08:24.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.833+0000 7f1b75d9b640 1 -- 172.21.15.170:0/2018109361 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 900349145 0 0) 0x7f1b7002d470 con 0x7f1b70035620 2024-05-08T04:08:24.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.833+0000 7f1b75d9b640 1 -- 172.21.15.170:0/2018109361 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b78051630 con 0x7f1b70035620 2024-05-08T04:08:24.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.833+0000 7f1b76d9d640 1 -- 172.21.15.170:0/2018109361 >> v1:172.21.15.187:6789/0 conn(0x7f1b700c45b0 legacy=0x7f1b700c4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:24.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.833+0000 7f1b75d9b640 1 -- 172.21.15.170:0/2018109361 <== mon.0 v1:172.21.15.170:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3314078967 0 0) 0x7f1b78051630 con 0x7f1b70035620 2024-05-08T04:08:24.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.833+0000 7f1b76d9d640 1 -- 172.21.15.170:0/2018109361 >> v1:172.21.15.170:6789/0 conn(0x7f1b70035620 legacy=0x7f1b70031e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:24.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.833+0000 7f1b76d9d640 1 -- 172.21.15.170:0/2018109361 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b700c1c30 con 0x7f1b70035240 2024-05-08T04:08:24.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.856+0000 7f533b7fe640 1 -- 172.21.15.170:0/745347369 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5354071910 con 0x7f535406fc80 2024-05-08T04:08:24.863 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.836+0000 7f1b76d9d640 1 --2- 172.21.15.170:0/2018109361 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1b70035240 0x7f1b700c16f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c0097d0 tx=0x7f1b6c02b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a006a06e26055b22 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:24.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.837+0000 7f1b75d9b640 1 -- 172.21.15.170:0/2018109361 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c037070 con 0x7f1b70035240 2024-05-08T04:08:24.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.837+0000 7f1b75d9b640 1 -- 172.21.15.170:0/2018109361 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b6c030510 con 0x7f1b70035240 2024-05-08T04:08:24.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.838+0000 7f1b7c816640 1 -- 172.21.15.170:0/2018109361 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1b70035240 msgr2=0x7f1b700c16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:24.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.838+0000 7f1b7c816640 1 --2- 172.21.15.170:0/2018109361 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1b70035240 0x7f1b700c16f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c0097d0 tx=0x7f1b6c02b770 comp rx=0 tx=0).stop 2024-05-08T04:08:24.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.838+0000 7f1b7c816640 1 -- 172.21.15.170:0/2018109361 shutdown_connections 2024-05-08T04:08:24.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.838+0000 7f1b7c816640 1 --2- 172.21.15.170:0/2018109361 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1b70035240 0x7f1b700c16f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.838+0000 7f1b7c816640 1 -- 172.21.15.170:0/2018109361 >> 172.21.15.170:0/2018109361 conn(0x7f1b70006aa0 msgr2=0x7f1b70006eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:24.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.839+0000 7f1b7c816640 1 -- 172.21.15.170:0/2018109361 shutdown_connections 2024-05-08T04:08:24.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.839+0000 7f1b7c816640 1 -- 172.21.15.170:0/2018109361 wait complete. 2024-05-08T04:08:24.865 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.841+0000 7f1b7c816640 1 Processor -- start 2024-05-08T04:08:24.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.841+0000 7f1b7c816640 1 -- start start 2024-05-08T04:08:24.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.842+0000 7f1b7c816640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b70035240 0x7f1b7002bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.842+0000 7f1b7c816640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1b700c45b0 0x7f1b70006210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.842+0000 7f1b7c816640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b7002a470 0x7f1b7002d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.842+0000 7f1b7c816640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1b700cfe00 con 0x7f1b7002a470 2024-05-08T04:08:24.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.842+0000 7f1b7c816640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1b700cfc80 con 0x7f1b70035240 2024-05-08T04:08:24.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.842+0000 7f1b7c816640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f1b700cff80 con 0x7f1b700c45b0 2024-05-08T04:08:24.866 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.842+0000 7f1b7659c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1b700c45b0 0x7f1b70006210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:24.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.842+0000 7f1b7659c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1b700c45b0 0x7f1b70006210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:54524/0 (socket says 172.21.15.170:54524) 2024-05-08T04:08:24.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.842+0000 7f1b7659c640 1 -- 172.21.15.170:0/3075211581 learned_addr learned my addr 172.21.15.170:0/3075211581 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:24.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.842+0000 7f1b7759e640 1 --2- 172.21.15.170:0/3075211581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b7002a470 0x7f1b7002d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:24.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.843+0000 7f1b7759e640 1 -- 172.21.15.170:0/3075211581 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1b700c45b0 msgr2=0x7f1b70006210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:24.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.843+0000 7f1b7759e640 1 --2- 172.21.15.170:0/3075211581 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1b700c45b0 0x7f1b70006210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.843+0000 7f1b7759e640 1 -- 172.21.15.170:0/3075211581 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b70035240 msgr2=0x7f1b7002bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:24.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.843+0000 7f1b7759e640 1 --2- 172.21.15.170:0/3075211581 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b70035240 0x7f1b7002bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.843+0000 7f1b7759e640 1 -- 172.21.15.170:0/3075211581 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b70029b20 con 0x7f1b7002a470 2024-05-08T04:08:24.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.843+0000 7f1b7759e640 1 --2- 172.21.15.170:0/3075211581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b7002a470 0x7f1b7002d920 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1b780755e0 tx=0x7f1b78069700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:24.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.844+0000 7f1b5f7fe640 1 -- 172.21.15.170:0/3075211581 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b78069ec0 con 0x7f1b7002a470 2024-05-08T04:08:24.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.844+0000 7f1b5f7fe640 1 -- 172.21.15.170:0/3075211581 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b780700d0 con 0x7f1b7002a470 2024-05-08T04:08:24.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.844+0000 7f1b5f7fe640 1 -- 172.21.15.170:0/3075211581 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b78070c60 con 0x7f1b7002a470 2024-05-08T04:08:24.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.844+0000 7f1b7c816640 1 -- 172.21.15.170:0/3075211581 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b7002b560 con 0x7f1b7002a470 2024-05-08T04:08:24.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.844+0000 7f1b7c816640 1 -- 172.21.15.170:0/3075211581 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b7002de60 con 0x7f1b7002a470 2024-05-08T04:08:24.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.848+0000 7f1b5f7fe640 1 -- 172.21.15.170:0/3075211581 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f1b78074130 con 0x7f1b7002a470 2024-05-08T04:08:24.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.849+0000 7f1b5f7fe640 1 --2- 172.21.15.170:0/3075211581 >> v2:172.21.15.187:6816/101007897 conn(0x7f1b640983a0 0x7f1b6409a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.849+0000 7f1b5f7fe640 1 -- 172.21.15.170:0/3075211581 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4090+0+0 (secure 0 0 0) 0x7f1b781126c0 con 0x7f1b7002a470 2024-05-08T04:08:24.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.849+0000 7f1b76d9d640 1 --2- 172.21.15.170:0/3075211581 >> v2:172.21.15.187:6816/101007897 conn(0x7f1b640983a0 0x7f1b6409a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:24.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.851+0000 7f1b76d9d640 1 --2- 172.21.15.170:0/3075211581 >> v2:172.21.15.187:6816/101007897 conn(0x7f1b640983a0 0x7f1b6409a860 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c034000 tx=0x7f1b6c02f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:24.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.851+0000 7f1b5d7fa640 1 -- 172.21.15.170:0/3075211581 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b58005080 con 0x7f1b7002a470 2024-05-08T04:08:24.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.861+0000 7f1b5f7fe640 1 -- 172.21.15.170:0/3075211581 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1b78099030 con 0x7f1b7002a470 2024-05-08T04:08:24.886 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.884+0000 7f50938ed640 1 Processor -- start 2024-05-08T04:08:24.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.885+0000 7f50938ed640 1 -- start start 2024-05-08T04:08:24.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.886+0000 7f50938ed640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5084037290 0x7f50840c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.886+0000 7f50938ed640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f5084037670 0x7f50840c4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.886+0000 7f50938ed640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50840c8ce0 con 0x7f50840c7df0 2024-05-08T04:08:24.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.886+0000 7f5091e8a640 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f50840c7df0 0x7f50840305a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:56002/0 (socket says 172.21.15.170:56002) 2024-05-08T04:08:24.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.886+0000 7f5091e8a640 1 -- 172.21.15.170:0/3951335805 learned_addr learned my addr 172.21.15.170:0/3951335805 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-05-08T04:08:24.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.886+0000 7f50938ed640 1 -- 172.21.15.170:0/3951335805 --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7f5084030cb0 con 0x7f5084037670 2024-05-08T04:08:24.888 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.886+0000 7f50938ed640 1 -- 172.21.15.170:0/3951335805 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f5084034050 con 0x7f5084037290 2024-05-08T04:08:24.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.887+0000 7f5090e88640 1 --2- 172.21.15.170:0/3951335805 >> v2:172.21.15.170:3300/0 conn(0x7f5084037670 0x7f50840c4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:24.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.887+0000 7f5091689640 1 --2- 172.21.15.170:0/3951335805 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5084037290 0x7f50840c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:24.889 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.887+0000 7f5090e88640 1 -- 172.21.15.170:0/3951335805 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5084037290 msgr2=0x7f50840c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:24.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.886+0000 7f96db59e640 1 Processor -- start 2024-05-08T04:08:24.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.886+0000 7f96db59e640 1 -- start start 2024-05-08T04:08:24.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.887+0000 7f96db59e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f96d4007190 0x7f96d4009260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.887+0000 7f5090e88640 1 --2- 172.21.15.170:0/3951335805 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5084037290 0x7f50840c87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.887+0000 7f5090e88640 1 -- 172.21.15.170:0/3951335805 >> v1:172.21.15.170:6789/0 conn(0x7f50840c7df0 legacy=0x7f50840305a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:24.890 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.887+0000 7f5090e88640 1 -- 172.21.15.170:0/3951335805 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5084034330 con 0x7f5084037670 2024-05-08T04:08:24.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.887+0000 7f96db59e640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f96d40338f0 0x7f96d4036b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.887+0000 7f96da59c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f96d4007190 0x7f96d4009260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:24.891 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.887+0000 7f96db59e640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96d40329c0 con 0x7f96d40097a0 2024-05-08T04:08:24.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.887+0000 7f96db59e640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7f96d4007570 con 0x7f96d40338f0 2024-05-08T04:08:24.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.887+0000 7f96db59e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f96d4032c30 con 0x7f96d4007190 2024-05-08T04:08:24.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.887+0000 7f96da59c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f96d4007190 0x7f96d4009260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:54536/0 (socket says 172.21.15.170:54536) 2024-05-08T04:08:24.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.887+0000 7f96da59c640 1 -- 172.21.15.170:0/2368853158 learned_addr learned my addr 172.21.15.170:0/2368853158 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:24.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.888+0000 7f96da59c640 1 -- 172.21.15.170:0/2368853158 >> v2:172.21.15.170:3300/0 conn(0x7f96d40338f0 msgr2=0x7f96d4036b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:24.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.888+0000 7f96da59c640 1 --2- 172.21.15.170:0/2368853158 >> v2:172.21.15.170:3300/0 conn(0x7f96d40338f0 0x7f96d4036b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.888+0000 7f96da59c640 1 -- 172.21.15.170:0/2368853158 >> v1:172.21.15.170:6789/0 conn(0x7f96d40097a0 legacy=0x7f96d4037070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:24.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.888+0000 7f96da59c640 1 -- 172.21.15.170:0/2368853158 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d4032f10 con 0x7f96d4007190 2024-05-08T04:08:24.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.888+0000 7f5091689640 1 --2- 172.21.15.170:0/3951335805 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5084037290 0x7f50840c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:08:24.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.889+0000 7fd252117640 1 Processor -- start 2024-05-08T04:08:24.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.889+0000 7f3c137e9640 1 Processor -- start 2024-05-08T04:08:24.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.889+0000 7f5090e88640 1 --2- 172.21.15.170:0/3951335805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5084037670 0x7f50840c4890 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f508c06a740 tx=0x7f508c0a9cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd7024f1c592ef8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:24.893 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.889+0000 7f96da59c640 1 --2- 172.21.15.170:0/2368853158 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f96d4007190 0x7f96d4009260 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009870 tx=0x7f96cc02b850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=120c8d74a403d557 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:24.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.889+0000 7f3c137e9640 1 -- start start 2024-05-08T04:08:24.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.889+0000 7f96d959a640 1 -- 172.21.15.170:0/2368853158 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc032640 con 0x7f96d4007190 2024-05-08T04:08:24.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.889+0000 7f96d959a640 1 -- 172.21.15.170:0/2368853158 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96cc037070 con 0x7f96d4007190 2024-05-08T04:08:24.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.889+0000 7f96d959a640 1 -- 172.21.15.170:0/2368853158 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc032a00 con 0x7f96d4007190 2024-05-08T04:08:24.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.889+0000 7fd252117640 1 -- start start 2024-05-08T04:08:24.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.890+0000 7f5083fff640 1 -- 172.21.15.170:0/3951335805 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c070660 con 0x7f5084037670 2024-05-08T04:08:24.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.890+0000 7f5083fff640 1 -- 172.21.15.170:0/3951335805 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f508c070800 con 0x7f5084037670 2024-05-08T04:08:24.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.890+0000 7f5083fff640 1 -- 172.21.15.170:0/3951335805 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f508c070bf0 con 0x7f5084037670 2024-05-08T04:08:24.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.890+0000 7f0b58ba8640 1 Processor -- start 2024-05-08T04:08:24.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.890+0000 7f96db59e640 1 -- 172.21.15.170:0/2368853158 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f96d4007190 msgr2=0x7f96d4009260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:24.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.890+0000 7f3c137e9640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3c0c0b06c0 0x7f3c0c0afc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.890+0000 7f3c137e9640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3c0c143090 0x7f3c0c1434e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.896 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.890+0000 7f3c137e9640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c0c0b0aa0 con 0x7f3c0c139500 2024-05-08T04:08:24.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.890+0000 7f3c137e9640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7f3c0c074070 con 0x7f3c0c143090 2024-05-08T04:08:24.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.890+0000 7f3c137e9640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3c0c13d650 con 0x7f3c0c0b06c0 2024-05-08T04:08:24.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.891+0000 7f3c11d86640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3c0c143090 0x7f3c0c1434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:24.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.891+0000 7f3c11d86640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3c0c143090 0x7f3c0c1434e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49262/0 (socket says 172.21.15.170:49262) 2024-05-08T04:08:24.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.891+0000 7f3c11d86640 1 -- 172.21.15.170:0/2253155558 learned_addr learned my addr 172.21.15.170:0/2253155558 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:24.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.890+0000 7f96db59e640 1 --2- 172.21.15.170:0/2368853158 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f96d4007190 0x7f96d4009260 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009870 tx=0x7f96cc02b850 comp rx=0 tx=0).stop 2024-05-08T04:08:24.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.891+0000 7f50938ed640 1 -- 172.21.15.170:0/3951335805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5084037670 msgr2=0x7f50840c4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:24.897 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.891+0000 7f50938ed640 1 --2- 172.21.15.170:0/3951335805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5084037670 0x7f50840c4890 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f508c06a740 tx=0x7f508c0a9cb0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.890+0000 7f0b58ba8640 1 -- start start 2024-05-08T04:08:24.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.891+0000 7f0b58ba8640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0b4c02dcb0 0x7f0b4c02efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.891+0000 7f0b58ba8640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b4c02c4d0 con 0x7f0b4c034540 2024-05-08T04:08:24.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.891+0000 7f0b58ba8640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b4c02c8b0 con 0x7f0b4c02d1e0 2024-05-08T04:08:24.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.891+0000 7f0b58ba8640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f0b4c034920 con 0x7f0b4c02dcb0 2024-05-08T04:08:24.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.890+0000 7fd252117640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd24c0b2450 0x7fd24c075ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.890+0000 7fd252117640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd24c144f70 0x7fd24c1453e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.890+0000 7fd252117640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd24c145920 con 0x7fd24c073f50 2024-05-08T04:08:24.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.890+0000 7fd252117640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd24c074330 con 0x7fd24c0b2450 2024-05-08T04:08:24.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.890+0000 7fd252117640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fd24c0714c0 con 0x7fd24c144f70 2024-05-08T04:08:24.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.891+0000 7fd24affd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd24c0b2450 0x7fd24c075ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:24.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.891+0000 7fd24affd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd24c0b2450 0x7fd24c075ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.170:47252/0 (socket says 172.21.15.170:47252) 2024-05-08T04:08:24.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.891+0000 7fd24affd640 1 -- 172.21.15.170:0/622933961 learned_addr learned my addr 172.21.15.170:0/622933961 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:24.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.891+0000 7fd24bfff640 1 --2- 172.21.15.170:0/622933961 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd24c144f70 0x7fd24c1453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:24.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.891+0000 7fd24affd640 1 -- 172.21.15.170:0/622933961 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd24c144f70 msgr2=0x7fd24c1453e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:24.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.891+0000 7fd24affd640 1 --2- 172.21.15.170:0/622933961 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd24c144f70 0x7fd24c1453e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.891+0000 7fd24affd640 1 -- 172.21.15.170:0/622933961 >> v1:172.21.15.187:6789/0 conn(0x7fd24c073f50 legacy=0x7fd24c072b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:24.899 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.891+0000 7fd24affd640 1 -- 172.21.15.170:0/622933961 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd24c073230 con 0x7fd24c0b2450 2024-05-08T04:08:24.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.891+0000 7f50938ed640 1 -- 172.21.15.170:0/3951335805 shutdown_connections 2024-05-08T04:08:24.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.891+0000 7f50938ed640 1 --2- 172.21.15.170:0/3951335805 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5084037670 0x7f50840c4890 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.891+0000 7f50938ed640 1 --2- 172.21.15.170:0/3951335805 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5084037290 0x7f50840c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.891+0000 7f50938ed640 1 -- 172.21.15.170:0/3951335805 >> 172.21.15.170:0/3951335805 conn(0x7f508403a730 msgr2=0x7f5084036b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:24.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.891+0000 7f96db59e640 1 -- 172.21.15.170:0/2368853158 shutdown_connections 2024-05-08T04:08:24.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.891+0000 7f96db59e640 1 --2- 172.21.15.170:0/2368853158 >> v2:172.21.15.170:3300/0 conn(0x7f96d40338f0 0x7f96d4036b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.891+0000 7f96db59e640 1 --2- 172.21.15.170:0/2368853158 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f96d4007190 0x7f96d4009260 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.900 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.891+0000 7f96db59e640 1 -- 172.21.15.170:0/2368853158 >> 172.21.15.170:0/2368853158 conn(0x7f96d40061f0 msgr2=0x7f96d4006600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:24.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.891+0000 7f0b53fff640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f0b4c02d1e0 0x7f0b4c0ce5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.170:50422/0 (socket says 172.21.15.170:50422) 2024-05-08T04:08:24.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.891+0000 7f0b53fff640 1 -- 172.21.15.170:0/3077439267 learned_addr learned my addr 172.21.15.170:0/3077439267 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-05-08T04:08:24.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.891+0000 7f3c11d86640 1 -- 172.21.15.170:0/2253155558 >> v2:172.21.15.187:3300/0 conn(0x7f3c0c0b06c0 msgr2=0x7f3c0c0afc50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T04:08:24.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.891+0000 7f3c11d86640 1 --2- 172.21.15.170:0/2253155558 >> v2:172.21.15.187:3300/0 conn(0x7f3c0c0b06c0 0x7f3c0c0afc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.891+0000 7f3c11d86640 1 -- 172.21.15.170:0/2253155558 >> v1:172.21.15.187:6789/0 conn(0x7f3c0c139500 legacy=0x7f3c0c139950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:24.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.891+0000 7f3c11d86640 1 -- 172.21.15.170:0/2253155558 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c0c073b50 con 0x7f3c0c143090 2024-05-08T04:08:24.901 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.891+0000 7f0b52ffd640 1 --2- 172.21.15.170:0/3077439267 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0b4c02dcb0 0x7f0b4c02efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:24.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.892+0000 7f50938ed640 1 -- 172.21.15.170:0/3951335805 shutdown_connections 2024-05-08T04:08:24.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.892+0000 7f50938ed640 1 -- 172.21.15.170:0/3951335805 wait complete. 2024-05-08T04:08:24.902 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.892+0000 7fd24bfff640 1 --2- 172.21.15.170:0/622933961 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd24c144f70 0x7fd24c1453e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:08:24.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.892+0000 7fd24affd640 1 --2- 172.21.15.170:0/622933961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd24c0b2450 0x7fd24c075ec0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd244009e50 tx=0x7fd24402bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=388e7d2d433f34f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:24.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.892+0000 7f3c11d86640 1 --2- 172.21.15.170:0/2253155558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c0c143090 0x7f3c0c1434e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0b1290 tx=0x7f3c0802bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b748600f04399e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:24.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.892+0000 7f96db59e640 1 -- 172.21.15.170:0/2368853158 shutdown_connections 2024-05-08T04:08:24.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.892+0000 7f96db59e640 1 -- 172.21.15.170:0/2368853158 wait complete. 2024-05-08T04:08:24.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.893+0000 7f50938ed640 1 Processor -- start 2024-05-08T04:08:24.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.892+0000 7f0b52ffd640 1 -- 172.21.15.170:0/3077439267 >> v1:172.21.15.187:6789/0 conn(0x7f0b4c02d1e0 legacy=0x7f0b4c0ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:24.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.892+0000 7f0b52ffd640 1 -- 172.21.15.170:0/3077439267 >> v1:172.21.15.170:6789/0 conn(0x7f0b4c034540 legacy=0x7f0b4c0c4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:24.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.892+0000 7f0b52ffd640 1 -- 172.21.15.170:0/3077439267 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b4c02fd70 con 0x7f0b4c02dcb0 2024-05-08T04:08:24.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.893+0000 7f3c03fff640 1 -- 172.21.15.170:0/2253155558 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0802d6b0 con 0x7f3c0c143090 2024-05-08T04:08:24.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.893+0000 7f3c03fff640 1 -- 172.21.15.170:0/2253155558 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c0802d850 con 0x7f3c0c143090 2024-05-08T04:08:24.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.893+0000 7f3c03fff640 1 -- 172.21.15.170:0/2253155558 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0802dc20 con 0x7f3c0c143090 2024-05-08T04:08:24.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.893+0000 7fd24a7fc640 1 -- 172.21.15.170:0/622933961 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd24402cb20 con 0x7fd24c0b2450 2024-05-08T04:08:24.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.893+0000 7fd24a7fc640 1 -- 172.21.15.170:0/622933961 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd24402ccc0 con 0x7fd24c0b2450 2024-05-08T04:08:24.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.894+0000 7fd252117640 1 -- 172.21.15.170:0/622933961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd24c0b2450 msgr2=0x7fd24c075ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:24.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.894+0000 7fd252117640 1 --2- 172.21.15.170:0/622933961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd24c0b2450 0x7fd24c075ec0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd244009e50 tx=0x7fd24402bf80 comp rx=0 tx=0).stop 2024-05-08T04:08:24.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.893+0000 7f3e5cdc8640 1 Processor -- start 2024-05-08T04:08:24.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.893+0000 7f3e5cdc8640 1 -- start start 2024-05-08T04:08:24.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.894+0000 7f0b52ffd640 1 --2- 172.21.15.170:0/3077439267 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0b4c02dcb0 0x7f0b4c02efd0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0b400097d0 tx=0x7f0b4002c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a460e309187315 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:24.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.893+0000 7f50938ed640 1 -- start start 2024-05-08T04:08:24.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.894+0000 7f50938ed640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5084037290 0x7f508402b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.894+0000 7f50938ed640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f508402b600 0x7f508416a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.894+0000 7f50938ed640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f508416a730 0x7f50840bfd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.894+0000 7f3c137e9640 1 -- 172.21.15.170:0/2253155558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c0c143090 msgr2=0x7f3c0c1434e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:24.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.894+0000 7f0b527fc640 1 -- 172.21.15.170:0/3077439267 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b40036020 con 0x7f0b4c02dcb0 2024-05-08T04:08:24.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.894+0000 7f3c137e9640 1 --2- 172.21.15.170:0/2253155558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c0c143090 0x7f3c0c1434e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c0b1290 tx=0x7f3c0802bb00 comp rx=0 tx=0).stop 2024-05-08T04:08:24.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.894+0000 7f5091e8a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f508416a730 0x7f50840bfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:24.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.894+0000 7f96db59e640 1 Processor -- start 2024-05-08T04:08:24.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.894+0000 7f96db59e640 1 -- start start 2024-05-08T04:08:24.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.894+0000 7f0b527fc640 1 -- 172.21.15.170:0/3077439267 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b40032640 con 0x7f0b4c02dcb0 2024-05-08T04:08:24.908 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.894+0000 7f5091e8a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f508416a730 0x7f50840bfd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49266/0 (socket says 172.21.15.170:49266) 2024-05-08T04:08:24.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.895+0000 7f3c137e9640 1 -- 172.21.15.170:0/2253155558 shutdown_connections 2024-05-08T04:08:24.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.895+0000 7f3c137e9640 1 --2- 172.21.15.170:0/2253155558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c0c143090 0x7f3c0c1434e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.895+0000 7f3c137e9640 1 --2- 172.21.15.170:0/2253155558 >> v2:172.21.15.187:3300/0 conn(0x7f3c0c0b06c0 0x7f3c0c0afc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.909 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.895+0000 7f3c137e9640 1 -- 172.21.15.170:0/2253155558 >> 172.21.15.170:0/2253155558 conn(0x7f3c0c0b5250 msgr2=0x7f3c0c0b5660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:24.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.895+0000 7f5090e88640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f508402b600 0x7f508416a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:24.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.895+0000 7f5090e88640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f508402b600 0x7f508416a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.170:47270/0 (socket says 172.21.15.170:47270) 2024-05-08T04:08:24.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.895+0000 7f5091e8a640 1 -- 172.21.15.170:0/2036052514 learned_addr learned my addr 172.21.15.170:0/2036052514 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:24.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.895+0000 7f50938ed640 1 -- 172.21.15.170:0/2036052514 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f508402d8e0 con 0x7f508416a730 2024-05-08T04:08:24.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.895+0000 7f50938ed640 1 -- 172.21.15.170:0/2036052514 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f50840c4dd0 con 0x7f508402b600 2024-05-08T04:08:24.910 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.895+0000 7f96db59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96d4007190 0x7f96d40a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.895+0000 7f96db59e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f96d40097a0 0x7f96d40a5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.895+0000 7f96db59e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96d40338f0 0x7f96d4151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.895+0000 7f96db59e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f96d4037780 con 0x7f96d4007190 2024-05-08T04:08:24.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.895+0000 7f96db59e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f96d40334d0 con 0x7f96d40338f0 2024-05-08T04:08:24.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.895+0000 7f96db59e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f96d4037900 con 0x7f96d40097a0 2024-05-08T04:08:24.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.895+0000 7f50938ed640 1 -- 172.21.15.170:0/2036052514 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f5084006aa0 con 0x7f5084037290 2024-05-08T04:08:24.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.895+0000 7f5091689640 1 --2- 172.21.15.170:0/2036052514 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5084037290 0x7f508402b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:24.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.895+0000 7f96d9d9b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f96d40097a0 0x7f96d40a5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:24.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.895+0000 7f96d9d9b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f96d40097a0 0x7f96d40a5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:54578/0 (socket says 172.21.15.170:54578) 2024-05-08T04:08:24.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.895+0000 7f96d9d9b640 1 -- 172.21.15.170:0/225136705 learned_addr learned my addr 172.21.15.170:0/225136705 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:24.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.895+0000 7f96da59c640 1 --2- 172.21.15.170:0/225136705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96d4007190 0x7f96d40a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:24.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.895+0000 7f96dad9d640 1 --2- 172.21.15.170:0/225136705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96d40338f0 0x7f96d4151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:24.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.895+0000 7f0b527fc640 1 -- 172.21.15.170:0/3077439267 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4002e040 con 0x7f0b4c02dcb0 2024-05-08T04:08:24.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.895+0000 7f0b58ba8640 1 -- 172.21.15.170:0/3077439267 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0b4c02dcb0 msgr2=0x7f0b4c02efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:24.912 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.895+0000 7f0b58ba8640 1 --2- 172.21.15.170:0/3077439267 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0b4c02dcb0 0x7f0b4c02efd0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0b400097d0 tx=0x7f0b4002c970 comp rx=0 tx=0).stop 2024-05-08T04:08:24.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.894+0000 7f3e5cdc8640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3e500c4090 0x7f3e500078d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.894+0000 7f3e5cdc8640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3e5002fe00 0x7f3e500c8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.894+0000 7f3e5cdc8640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e500334f0 con 0x7f3e50032400 2024-05-08T04:08:24.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.894+0000 7f3e5cdc8640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3e50006950 con 0x7f3e5002fe00 2024-05-08T04:08:24.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.894+0000 7f3e5cdc8640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f3e500c1d00 con 0x7f3e500c4090 2024-05-08T04:08:24.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.895+0000 7f3e577fe640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3e500c4090 0x7f3e500078d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:24.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.895+0000 7f3e577fe640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3e500c4090 0x7f3e500078d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:54562/0 (socket says 172.21.15.170:54562) 2024-05-08T04:08:24.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.895+0000 7f3e577fe640 1 -- 172.21.15.170:0/2791391670 learned_addr learned my addr 172.21.15.170:0/2791391670 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:24.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.895+0000 7f3e577fe640 1 -- 172.21.15.170:0/2791391670 >> v2:172.21.15.187:3300/0 conn(0x7f3e5002fe00 msgr2=0x7f3e500c8930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T04:08:24.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.895+0000 7f3e577fe640 1 --2- 172.21.15.170:0/2791391670 >> v2:172.21.15.187:3300/0 conn(0x7f3e5002fe00 0x7f3e500c8930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.913 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.895+0000 7f3e577fe640 1 -- 172.21.15.170:0/2791391670 >> v1:172.21.15.187:6789/0 conn(0x7f3e50032400 legacy=0x7f3e50006240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:24.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.896+0000 7fd252117640 1 -- 172.21.15.170:0/622933961 shutdown_connections 2024-05-08T04:08:24.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.896+0000 7fd252117640 1 --2- 172.21.15.170:0/622933961 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd24c144f70 0x7fd24c1453e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.896+0000 7fd252117640 1 --2- 172.21.15.170:0/622933961 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd24c0b2450 0x7fd24c075ec0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.896+0000 7fd252117640 1 -- 172.21.15.170:0/622933961 >> 172.21.15.170:0/622933961 conn(0x7fd24c0b5fb0 msgr2=0x7fd24c0b63c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:24.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.896+0000 7fd252117640 1 -- 172.21.15.170:0/622933961 shutdown_connections 2024-05-08T04:08:24.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.896+0000 7f96da59c640 1 -- 172.21.15.170:0/225136705 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f96d40097a0 msgr2=0x7f96d40a5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:24.914 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.896+0000 7f96da59c640 1 --2- 172.21.15.170:0/225136705 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f96d40097a0 0x7f96d40a5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.896+0000 7f96da59c640 1 -- 172.21.15.170:0/225136705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96d40338f0 msgr2=0x7f96d4151d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:24.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.896+0000 7f96da59c640 1 --2- 172.21.15.170:0/225136705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96d40338f0 0x7f96d4151d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.896+0000 7f96da59c640 1 -- 172.21.15.170:0/225136705 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d40a4fd0 con 0x7f96d4007190 2024-05-08T04:08:24.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.896+0000 7f3c137e9640 1 -- 172.21.15.170:0/2253155558 shutdown_connections 2024-05-08T04:08:24.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.896+0000 7f0b58ba8640 1 -- 172.21.15.170:0/3077439267 shutdown_connections 2024-05-08T04:08:24.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.896+0000 7f0b58ba8640 1 --2- 172.21.15.170:0/3077439267 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0b4c02dcb0 0x7f0b4c02efd0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.915 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.896+0000 7f0b58ba8640 1 -- 172.21.15.170:0/3077439267 >> 172.21.15.170:0/3077439267 conn(0x7f0b4c036610 msgr2=0x7f0b4c005e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:24.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.896+0000 7f3c137e9640 1 -- 172.21.15.170:0/2253155558 wait complete. 2024-05-08T04:08:24.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.895+0000 7f3e577fe640 1 -- 172.21.15.170:0/2791391670 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e500337d0 con 0x7f3e500c4090 2024-05-08T04:08:24.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.896+0000 7f0b58ba8640 1 -- 172.21.15.170:0/3077439267 shutdown_connections 2024-05-08T04:08:24.916 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.896+0000 7f96da59c640 1 --2- 172.21.15.170:0/225136705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96d4007190 0x7f96d40a3780 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f96cc033000 tx=0x7f96cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:24.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.896+0000 7f5091e8a640 1 -- 172.21.15.170:0/2036052514 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5084037290 msgr2=0x7f508402b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:24.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.896+0000 7f5091e8a640 1 --2- 172.21.15.170:0/2036052514 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5084037290 0x7f508402b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.896+0000 7f5091e8a640 1 -- 172.21.15.170:0/2036052514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f508402b600 msgr2=0x7f508416a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:24.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.896+0000 7f5091e8a640 1 --2- 172.21.15.170:0/2036052514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f508402b600 0x7f508416a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.896+0000 7f5091e8a640 1 -- 172.21.15.170:0/2036052514 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f508402c360 con 0x7f508416a730 2024-05-08T04:08:24.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.896+0000 7f5091e8a640 1 --2- 172.21.15.170:0/2036052514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f508416a730 0x7f50840bfd50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f507c00c730 tx=0x7f507c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:24.917 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.896+0000 7f3e577fe640 1 --2- 172.21.15.170:0/2791391670 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3e500c4090 0x7f3e500078d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3e580520f0 tx=0x7f3e580b03c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ad8db55faedabe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:24.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.897+0000 7f5081ffb640 1 -- 172.21.15.170:0/2036052514 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c017070 con 0x7f508416a730 2024-05-08T04:08:24.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.897+0000 7f5081ffb640 1 -- 172.21.15.170:0/2036052514 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f507c005870 con 0x7f508416a730 2024-05-08T04:08:24.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.897+0000 7f50938ed640 1 -- 172.21.15.170:0/2036052514 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f508402c640 con 0x7f508416a730 2024-05-08T04:08:24.918 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.897+0000 7f50938ed640 1 -- 172.21.15.170:0/2036052514 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f508402cd40 con 0x7f508416a730 2024-05-08T04:08:24.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.897+0000 7f3e567fc640 1 -- 172.21.15.170:0/2791391670 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e580700d0 con 0x7f3e500c4090 2024-05-08T04:08:24.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.897+0000 7f3e567fc640 1 -- 172.21.15.170:0/2791391670 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e58070270 con 0x7f3e500c4090 2024-05-08T04:08:24.919 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.897+0000 7f3c137e9640 1 Processor -- start 2024-05-08T04:08:24.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.896+0000 7f0b58ba8640 1 -- 172.21.15.170:0/3077439267 wait complete. 2024-05-08T04:08:24.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.898+0000 7f0b58ba8640 1 Processor -- start 2024-05-08T04:08:24.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.898+0000 7f3e567fc640 1 -- 172.21.15.170:0/2791391670 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5806e670 con 0x7f3e500c4090 2024-05-08T04:08:24.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.896+0000 7fd252117640 1 -- 172.21.15.170:0/622933961 wait complete. 2024-05-08T04:08:24.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.898+0000 7f3e5cdc8640 1 -- 172.21.15.170:0/2791391670 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3e500c4090 msgr2=0x7f3e500078d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:24.920 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.898+0000 7f3e5cdc8640 1 --2- 172.21.15.170:0/2791391670 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3e500c4090 0x7f3e500078d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3e580520f0 tx=0x7f3e580b03c0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.898+0000 7f3c137e9640 1 -- start start 2024-05-08T04:08:24.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.898+0000 7f3c137e9640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3c0c0b06c0 0x7f3c0c13a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.921 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.898+0000 7f3c11585640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3c0c0b06c0 0x7f3c0c13a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:24.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.899+0000 7f3e5cdc8640 1 -- 172.21.15.170:0/2791391670 shutdown_connections 2024-05-08T04:08:24.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.899+0000 7f3e5cdc8640 1 --2- 172.21.15.170:0/2791391670 >> v2:172.21.15.187:3300/0 conn(0x7f3e5002fe00 0x7f3e500c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.899+0000 7f3e5cdc8640 1 --2- 172.21.15.170:0/2791391670 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3e500c4090 0x7f3e500078d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.899+0000 7f3e5cdc8640 1 -- 172.21.15.170:0/2791391670 >> 172.21.15.170:0/2791391670 conn(0x7f3e50006ad0 msgr2=0x7f3e50006ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:24.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.897+0000 7f5081ffb640 1 -- 172.21.15.170:0/2036052514 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c005c60 con 0x7f508416a730 2024-05-08T04:08:24.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.899+0000 7f5081ffb640 1 -- 172.21.15.170:0/2036052514 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f507c01a070 con 0x7f508416a730 2024-05-08T04:08:24.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.899+0000 7f96caffd640 1 -- 172.21.15.170:0/225136705 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc0455f0 con 0x7f96d4007190 2024-05-08T04:08:24.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.899+0000 7f96caffd640 1 -- 172.21.15.170:0/225136705 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96cc0457b0 con 0x7f96d4007190 2024-05-08T04:08:24.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.899+0000 7f96caffd640 1 -- 172.21.15.170:0/225136705 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96cc045ba0 con 0x7f96d4007190 2024-05-08T04:08:24.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.899+0000 7f96db59e640 1 -- 172.21.15.170:0/225136705 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96d40a3e10 con 0x7f96d4007190 2024-05-08T04:08:24.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.899+0000 7f96db59e640 1 -- 172.21.15.170:0/225136705 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96d40a2fc0 con 0x7f96d4007190 2024-05-08T04:08:24.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.899+0000 7fd252117640 1 Processor -- start 2024-05-08T04:08:24.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.900+0000 7f5081ffb640 1 --2- 172.21.15.170:0/2036052514 >> v2:172.21.15.187:6816/101007897 conn(0x7f5070076360 0x7f5070078820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.899+0000 7f3c11585640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3c0c0b06c0 0x7f3c0c13a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:54588/0 (socket says 172.21.15.170:54588) 2024-05-08T04:08:24.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.898+0000 7f3c137e9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c0c139500 0x7f3c0c13f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.899+0000 7f3c137e9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c0c143090 0x7f3c0c13f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.899+0000 7f3c137e9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c0b0d40 con 0x7f3c0c139500 2024-05-08T04:08:24.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.899+0000 7f3c137e9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c074440 con 0x7f3c0c143090 2024-05-08T04:08:24.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.899+0000 7f3c137e9640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f3c0c0b0ec0 con 0x7f3c0c0b06c0 2024-05-08T04:08:24.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.900+0000 7f3c11585640 1 -- 172.21.15.170:0/99774960 learned_addr learned my addr 172.21.15.170:0/99774960 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:24.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.898+0000 7f0b58ba8640 1 -- start start 2024-05-08T04:08:24.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.901+0000 7f3c11585640 1 -- 172.21.15.170:0/99774960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c0c143090 msgr2=0x7f3c0c13f640 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T04:08:24.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.901+0000 7f3c11585640 1 --2- 172.21.15.170:0/99774960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c0c143090 0x7f3c0c13f640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.901+0000 7f3c11585640 1 -- 172.21.15.170:0/99774960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c0c139500 msgr2=0x7f3c0c13f100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T04:08:24.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.901+0000 7f3c11585640 1 --2- 172.21.15.170:0/99774960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c0c139500 0x7f3c0c13f100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.925 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.901+0000 7f3c11585640 1 -- 172.21.15.170:0/99774960 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c0c13c300 con 0x7f3c0c0b06c0 2024-05-08T04:08:24.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.899+0000 7f0b58ba8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b4c02d1e0 0x7f0b4c02b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.899+0000 7f0b58ba8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b4c02dcb0 0x7f0b4c164b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.899+0000 7f0b58ba8640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0b4c034540 0x7f0b4c14eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.899+0000 7f0b53fff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0b4c034540 0x7f0b4c14eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:24.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.899+0000 7f0b53fff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0b4c034540 0x7f0b4c14eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:54598/0 (socket says 172.21.15.170:54598) 2024-05-08T04:08:24.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.899+0000 7f0b53fff640 1 -- 172.21.15.170:0/1262601489 learned_addr learned my addr 172.21.15.170:0/1262601489 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:24.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.899+0000 7f0b58ba8640 1 -- 172.21.15.170:0/1262601489 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4c033970 con 0x7f0b4c02d1e0 2024-05-08T04:08:24.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.899+0000 7f0b58ba8640 1 -- 172.21.15.170:0/1262601489 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4c02cd50 con 0x7f0b4c02dcb0 2024-05-08T04:08:24.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.899+0000 7f0b58ba8640 1 -- 172.21.15.170:0/1262601489 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f0b4c033af0 con 0x7f0b4c034540 2024-05-08T04:08:24.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.900+0000 7f0b53fff640 1 -- 172.21.15.170:0/1262601489 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b4c02dcb0 msgr2=0x7f0b4c164b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T04:08:24.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.900+0000 7f0b53fff640 1 --2- 172.21.15.170:0/1262601489 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b4c02dcb0 0x7f0b4c164b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.900+0000 7f0b53fff640 1 -- 172.21.15.170:0/1262601489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b4c02d1e0 msgr2=0x7f0b4c02b950 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T04:08:24.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.900+0000 7f0b53fff640 1 --2- 172.21.15.170:0/1262601489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b4c02d1e0 0x7f0b4c02b950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.900+0000 7f0b53fff640 1 -- 172.21.15.170:0/1262601489 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b4c0a1cd0 con 0x7f0b4c034540 2024-05-08T04:08:24.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.901+0000 7f0b53fff640 1 --2- 172.21.15.170:0/1262601489 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0b4c034540 0x7f0b4c14eb10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0b4800c820 tx=0x7f0b4800cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:24.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.901+0000 7f50938ed640 1 -- 172.21.15.170:0/2036052514 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50840057d0 con 0x7f508416a730 2024-05-08T04:08:24.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.901+0000 7f5081ffb640 1 -- 172.21.15.170:0/2036052514 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4090+0+0 (secure 0 0 0) 0x7f507c095fe0 con 0x7f508416a730 2024-05-08T04:08:24.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.901+0000 7f3c11585640 1 --2- 172.21.15.170:0/99774960 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3c0c0b06c0 0x7f3c0c13a7b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc00b410 tx=0x7f3bfc00b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:24.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.902+0000 7f3c01ffb640 1 -- 172.21.15.170:0/99774960 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc015020 con 0x7f3c0c0b06c0 2024-05-08T04:08:24.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.902+0000 7f96caffd640 1 -- 172.21.15.170:0/225136705 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f96cc04e440 con 0x7f96d4007190 2024-05-08T04:08:24.928 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.902+0000 7f5091689640 1 --2- 172.21.15.170:0/2036052514 >> v2:172.21.15.187:6816/101007897 conn(0x7f5070076360 0x7f5070078820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:24.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.900+0000 7fd252117640 1 -- start start 2024-05-08T04:08:24.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.900+0000 7fd252117640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd24c073f50 0x7fd24c0abbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.900+0000 7fd252117640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd24c0b2450 0x7fd24c1401c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.901+0000 7fd252117640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd24c144f70 0x7fd24c140700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.901+0000 7fd252117640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd24c070930 con 0x7fd24c144f70 2024-05-08T04:08:24.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.901+0000 7fd252117640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd24c0707b0 con 0x7fd24c073f50 2024-05-08T04:08:24.929 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.901+0000 7fd252117640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fd24c152590 con 0x7fd24c0b2450 2024-05-08T04:08:24.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.901+0000 7fd24bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd24c144f70 0x7fd24c140700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:24.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.901+0000 7fd24bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd24c144f70 0x7fd24c140700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49284/0 (socket says 172.21.15.170:49284) 2024-05-08T04:08:24.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.901+0000 7fd24bfff640 1 -- 172.21.15.170:0/1662155208 learned_addr learned my addr 172.21.15.170:0/1662155208 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:24.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.902+0000 7fd24bfff640 1 -- 172.21.15.170:0/1662155208 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd24c0b2450 msgr2=0x7fd24c1401c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T04:08:24.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.902+0000 7fd24affd640 1 --2- 172.21.15.170:0/1662155208 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd24c0b2450 0x7fd24c1401c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:24.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.902+0000 7fd24bfff640 1 --2- 172.21.15.170:0/1662155208 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd24c0b2450 0x7fd24c1401c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.902+0000 7fd24bfff640 1 -- 172.21.15.170:0/1662155208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd24c073f50 msgr2=0x7fd24c0abbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:24.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.902+0000 7fd24bfff640 1 --2- 172.21.15.170:0/1662155208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd24c073f50 0x7fd24c0abbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.902+0000 7fd24bfff640 1 -- 172.21.15.170:0/1662155208 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd24c0acf70 con 0x7fd24c144f70 2024-05-08T04:08:24.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.899+0000 7f3e5cdc8640 1 -- 172.21.15.170:0/2791391670 shutdown_connections 2024-05-08T04:08:24.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.899+0000 7f3e5cdc8640 1 -- 172.21.15.170:0/2791391670 wait complete. 2024-05-08T04:08:24.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.900+0000 7f3e5cdc8640 1 Processor -- start 2024-05-08T04:08:24.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.901+0000 7f3e5cdc8640 1 -- start start 2024-05-08T04:08:24.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.902+0000 7f3e5cdc8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e5002fe00 0x7f3e500bc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.902+0000 7f3e5cdc8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e50032400 0x7f3e5002a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.902+0000 7f3e5cdc8640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3e50161450 0x7f3e5002ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.902+0000 7f3e5cdc8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3e500c4470 con 0x7f3e5002fe00 2024-05-08T04:08:24.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.902+0000 7f3e5cdc8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3e50031e70 con 0x7f3e50032400 2024-05-08T04:08:24.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.902+0000 7f3e5cdc8640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f3e50007e10 con 0x7f3e50161450 2024-05-08T04:08:24.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.903+0000 7fd24bfff640 1 --2- 172.21.15.170:0/1662155208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd24c144f70 0x7fd24c140700 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00d5c0 tx=0x7fd23c00da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:24.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.902+0000 7f0b3ffff640 1 -- 172.21.15.170:0/1262601489 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b48018070 con 0x7f0b4c034540 2024-05-08T04:08:24.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.902+0000 7f0b3ffff640 1 -- 172.21.15.170:0/1262601489 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b48004b10 con 0x7f0b4c034540 2024-05-08T04:08:24.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.902+0000 7f0b3ffff640 1 -- 172.21.15.170:0/1262601489 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b480070d0 con 0x7f0b4c034540 2024-05-08T04:08:24.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.903+0000 7f0b58ba8640 1 -- 172.21.15.170:0/1262601489 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b4c14e600 con 0x7f0b4c034540 2024-05-08T04:08:24.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.903+0000 7f0b58ba8640 1 -- 172.21.15.170:0/1262601489 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b4c14e1a0 con 0x7f0b4c034540 2024-05-08T04:08:24.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.903+0000 7f3e577fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e5002fe00 0x7f3e500bc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:24.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.903+0000 7f3e577fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e5002fe00 0x7f3e500bc3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49300/0 (socket says 172.21.15.170:49300) 2024-05-08T04:08:24.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.903+0000 7f3e577fe640 1 -- 172.21.15.170:0/4114720189 learned_addr learned my addr 172.21.15.170:0/4114720189 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:24.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.903+0000 7f3e577fe640 1 -- 172.21.15.170:0/4114720189 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3e50161450 msgr2=0x7f3e5002ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:24.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.903+0000 7f3e577fe640 1 --2- 172.21.15.170:0/4114720189 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3e50161450 0x7f3e5002ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.903+0000 7f3e577fe640 1 -- 172.21.15.170:0/4114720189 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e50032400 msgr2=0x7f3e5002a710 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T04:08:24.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.903+0000 7f3e577fe640 1 --2- 172.21.15.170:0/4114720189 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e50032400 0x7f3e5002a710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:24.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.903+0000 7f3e577fe640 1 -- 172.21.15.170:0/4114720189 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e5014f7c0 con 0x7f3e5002fe00 2024-05-08T04:08:24.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.903+0000 7f3e577fe640 1 --2- 172.21.15.170:0/4114720189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e5002fe00 0x7f3e500bc3f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3e58065ed0 tx=0x7f3e58074cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:24.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.902+0000 7f3c01ffb640 1 -- 172.21.15.170:0/99774960 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bfc011830 con 0x7f3c0c0b06c0 2024-05-08T04:08:24.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.902+0000 7f3c137e9640 1 -- 172.21.15.170:0/99774960 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c0c13c5e0 con 0x7f3c0c0b06c0 2024-05-08T04:08:24.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.903+0000 7f3c137e9640 1 -- 172.21.15.170:0/99774960 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c0c13b410 con 0x7f3c0c0b06c0 2024-05-08T04:08:24.933 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.904+0000 7f3c01ffb640 1 -- 172.21.15.170:0/99774960 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc015020 con 0x7f3c0c0b06c0 2024-05-08T04:08:24.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.904+0000 7f5091689640 1 --2- 172.21.15.170:0/2036052514 >> v2:172.21.15.187:6816/101007897 conn(0x7f5070076360 0x7f5070078820 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f508800a990 tx=0x7f5088008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:24.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.904+0000 7f96caffd640 1 --2- 172.21.15.170:0/225136705 >> v2:172.21.15.187:6816/101007897 conn(0x7f96bc076390 0x7f96bc078850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.904+0000 7f96caffd640 1 -- 172.21.15.170:0/225136705 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4090+0+0 (secure 0 0 0) 0x7f96cc0b8e50 con 0x7f96d4007190 2024-05-08T04:08:24.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.904+0000 7f96d9d9b640 1 --2- 172.21.15.170:0/225136705 >> v2:172.21.15.187:6816/101007897 conn(0x7f96bc076390 0x7f96bc078850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:24.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.904+0000 7f96db59e640 1 -- 172.21.15.170:0/225136705 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96c0005080 con 0x7f96d4007190 2024-05-08T04:08:24.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.905+0000 7f0b58ba8640 1 -- 172.21.15.170:0/1262601489 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b4c005380 con 0x7f0b4c034540 2024-05-08T04:08:24.935 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.905+0000 7f96d9d9b640 1 --2- 172.21.15.170:0/225136705 >> v2:172.21.15.187:6816/101007897 conn(0x7f96bc076390 0x7f96bc078850 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f96dc074e40 tx=0x7f96dc06b300 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:24.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.905+0000 7f3c137e9640 1 -- 172.21.15.170:0/99774960 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c0c06b400 con 0x7f3c0c0b06c0 2024-05-08T04:08:24.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.905+0000 7f3c01ffb640 1 -- 172.21.15.170:0/99774960 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f3bfc018020 con 0x7f3c0c0b06c0 2024-05-08T04:08:24.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.906+0000 7f3c01ffb640 1 --2- 172.21.15.170:0/99774960 >> v2:172.21.15.187:6816/101007897 conn(0x7f3bf0076490 0x7f3bf0078950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.906+0000 7f3c01ffb640 1 -- 172.21.15.170:0/99774960 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 4090+0+0 (secure 0 0 0) 0x7f3bfc096ba0 con 0x7f3c0c0b06c0 2024-05-08T04:08:24.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.906+0000 7f3c10d84640 1 --2- 172.21.15.170:0/99774960 >> v2:172.21.15.187:6816/101007897 conn(0x7f3bf0076490 0x7f3bf0078950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:24.936 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.906+0000 7f0b3ffff640 1 -- 172.21.15.170:0/1262601489 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f0b4801b070 con 0x7f0b4c034540 2024-05-08T04:08:24.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.907+0000 7f3c10d84640 1 --2- 172.21.15.170:0/99774960 >> v2:172.21.15.187:6816/101007897 conn(0x7f3bf0076490 0x7f3bf0078950 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c137440 tx=0x7f3c04007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:24.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.907+0000 7f0b3ffff640 1 --2- 172.21.15.170:0/1262601489 >> v2:172.21.15.187:6816/101007897 conn(0x7f0b380762e0 0x7f0b380787a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.907+0000 7f0b3ffff640 1 -- 172.21.15.170:0/1262601489 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 4090+0+0 (secure 0 0 0) 0x7f0b480963f0 con 0x7f0b4c034540 2024-05-08T04:08:24.937 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.907+0000 7f0b537fe640 1 --2- 172.21.15.170:0/1262601489 >> v2:172.21.15.187:6816/101007897 conn(0x7f0b380762e0 0x7f0b380787a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:24.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.908+0000 7f0b537fe640 1 --2- 172.21.15.170:0/1262601489 >> v2:172.21.15.187:6816/101007897 conn(0x7f0b380762e0 0x7f0b380787a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0b5406c270 tx=0x7f0b54068050 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:24.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.911+0000 7f3c01ffb640 1 -- 172.21.15.170:0/99774960 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3bfc01f020 con 0x7f3c0c0b06c0 2024-05-08T04:08:24.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.911+0000 7fd233fff640 1 -- 172.21.15.170:0/1662155208 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c004030 con 0x7fd24c144f70 2024-05-08T04:08:24.938 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.911+0000 7f3e43fff640 1 -- 172.21.15.170:0/4114720189 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5806ee10 con 0x7f3e5002fe00 2024-05-08T04:08:24.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.911+0000 7f5081ffb640 1 -- 172.21.15.170:0/2036052514 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f507c05f9b0 con 0x7f508416a730 2024-05-08T04:08:24.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.911+0000 7fd252117640 1 -- 172.21.15.170:0/1662155208 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd24c0ad250 con 0x7fd24c144f70 2024-05-08T04:08:24.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.911+0000 7fd252117640 1 -- 172.21.15.170:0/1662155208 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd24c140c40 con 0x7fd24c144f70 2024-05-08T04:08:24.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.912+0000 7f3e5cdc8640 1 -- 172.21.15.170:0/4114720189 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e500bce70 con 0x7f3e5002fe00 2024-05-08T04:08:24.939 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.913+0000 7f3e5cdc8640 1 -- 172.21.15.170:0/4114720189 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e5002b370 con 0x7f3e5002fe00 2024-05-08T04:08:24.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.912+0000 7fd252117640 1 -- 172.21.15.170:0/1662155208 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd24c069bd0 con 0x7fd24c144f70 2024-05-08T04:08:24.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.914+0000 7f3e5cdc8640 1 -- 172.21.15.170:0/4114720189 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e500057e0 con 0x7f3e5002fe00 2024-05-08T04:08:24.940 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.920+0000 7f0b3ffff640 1 -- 172.21.15.170:0/1262601489 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0b4c0a1cd0 con 0x7f0b4c034540 2024-05-08T04:08:24.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.921+0000 7f3e43fff640 1 -- 172.21.15.170:0/4114720189 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e580955c0 con 0x7f3e5002fe00 2024-05-08T04:08:24.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.921+0000 7f3e43fff640 1 -- 172.21.15.170:0/4114720189 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e580b87f0 con 0x7f3e5002fe00 2024-05-08T04:08:24.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.921+0000 7f96caffd640 1 -- 172.21.15.170:0/225136705 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f96cc040070 con 0x7f96d4007190 2024-05-08T04:08:24.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.922+0000 7fd233fff640 1 -- 172.21.15.170:0/1662155208 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd23c0041d0 con 0x7fd24c144f70 2024-05-08T04:08:24.941 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.922+0000 7fd233fff640 1 -- 172.21.15.170:0/1662155208 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c003610 con 0x7fd24c144f70 2024-05-08T04:08:24.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.924+0000 7f3e43fff640 1 -- 172.21.15.170:0/4114720189 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f3e580b73a0 con 0x7f3e5002fe00 2024-05-08T04:08:24.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.925+0000 7f3e43fff640 1 --2- 172.21.15.170:0/4114720189 >> v2:172.21.15.187:6816/101007897 conn(0x7f3e3c0764c0 0x7f3e3c078980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.925+0000 7f3e43fff640 1 -- 172.21.15.170:0/4114720189 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4090+0+0 (secure 0 0 0) 0x7f3e580701b0 con 0x7f3e5002fe00 2024-05-08T04:08:24.942 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.925+0000 7fd233fff640 1 -- 172.21.15.170:0/1662155208 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7fd23c0037b0 con 0x7fd24c144f70 2024-05-08T04:08:24.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.926+0000 7f3e56ffd640 1 --2- 172.21.15.170:0/4114720189 >> v2:172.21.15.187:6816/101007897 conn(0x7f3e3c0764c0 0x7f3e3c078980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:24.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.926+0000 7fd233fff640 1 --2- 172.21.15.170:0/1662155208 >> v2:172.21.15.187:6816/101007897 conn(0x7fd238076290 0x7fd238078750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:24.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.926+0000 7fd233fff640 1 -- 172.21.15.170:0/1662155208 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4090+0+0 (secure 0 0 0) 0x7fd23c097c40 con 0x7fd24c144f70 2024-05-08T04:08:24.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.927+0000 7fd24b7fe640 1 --2- 172.21.15.170:0/1662155208 >> v2:172.21.15.187:6816/101007897 conn(0x7fd238076290 0x7fd238078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:24.943 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.928+0000 7fd24b7fe640 1 --2- 172.21.15.170:0/1662155208 >> v2:172.21.15.187:6816/101007897 conn(0x7fd238076290 0x7fd238078750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd2400015f0 tx=0x7fd240007970 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:24.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.928+0000 7f3e56ffd640 1 --2- 172.21.15.170:0/4114720189 >> v2:172.21.15.187:6816/101007897 conn(0x7f3e3c0764c0 0x7f3e3c078980 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3e50031b40 tx=0x7f3e4c007950 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:24.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.933+0000 7fd233fff640 1 -- 172.21.15.170:0/1662155208 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd23c061cd0 con 0x7fd24c144f70 2024-05-08T04:08:24.944 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:24.941+0000 7f3e43fff640 1 -- 172.21.15.170:0/4114720189 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3e580fd7d0 con 0x7f3e5002fe00 2024-05-08T04:08:25.276 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.274+0000 7f5353fff640 1 -- 172.21.15.170:0/745347369 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f5334005370 con 0x7f535406fc80 2024-05-08T04:08:25.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.275+0000 7f1b5d7fa640 1 -- 172.21.15.170:0/3075211581 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f1b58005370 con 0x7f1b7002a470 2024-05-08T04:08:25.277 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.275+0000 7f1b5f7fe640 1 -- 172.21.15.170:0/3075211581 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1b78090030 con 0x7f1b7002a470 2024-05-08T04:08:25.277 INFO:teuthology.orchestra.run.smithi170.stdout:60129542147 2024-05-08T04:08:25.278 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.276+0000 7f533b7fe640 1 -- 172.21.15.170:0/745347369 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f534c083650 con 0x7f535406fc80 2024-05-08T04:08:25.278 INFO:teuthology.orchestra.run.smithi170.stdout:60129542147 2024-05-08T04:08:25.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.279+0000 7f1b7c816640 1 -- 172.21.15.170:0/3075211581 >> v2:172.21.15.187:6816/101007897 conn(0x7f1b640983a0 msgr2=0x7f1b6409a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:25.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.279+0000 7f1b7c816640 1 --2- 172.21.15.170:0/3075211581 >> v2:172.21.15.187:6816/101007897 conn(0x7f1b640983a0 0x7f1b6409a860 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c034000 tx=0x7f1b6c02f000 comp rx=0 tx=0).stop 2024-05-08T04:08:25.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.279+0000 7f1b7c816640 1 -- 172.21.15.170:0/3075211581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b7002a470 msgr2=0x7f1b7002d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:25.281 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.279+0000 7f1b7c816640 1 --2- 172.21.15.170:0/3075211581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b7002a470 0x7f1b7002d920 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1b780755e0 tx=0x7f1b78069700 comp rx=0 tx=0).stop 2024-05-08T04:08:25.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.279+0000 7f1b7c816640 1 -- 172.21.15.170:0/3075211581 shutdown_connections 2024-05-08T04:08:25.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.279+0000 7f1b7c816640 1 --2- 172.21.15.170:0/3075211581 >> v2:172.21.15.187:6816/101007897 conn(0x7f1b640983a0 0x7f1b6409a860 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.279+0000 7f1b7c816640 1 --2- 172.21.15.170:0/3075211581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b7002a470 0x7f1b7002d920 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.279+0000 7f1b7c816640 1 --2- 172.21.15.170:0/3075211581 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1b700c45b0 0x7f1b70006210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.279+0000 7f1b7c816640 1 --2- 172.21.15.170:0/3075211581 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1b70035240 0x7f1b7002bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.280+0000 7f1b7c816640 1 -- 172.21.15.170:0/3075211581 >> 172.21.15.170:0/3075211581 conn(0x7f1b70006aa0 msgr2=0x7f1b70034980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:25.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.280+0000 7f1b7c816640 1 -- 172.21.15.170:0/3075211581 shutdown_connections 2024-05-08T04:08:25.282 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.280+0000 7f1b7c816640 1 -- 172.21.15.170:0/3075211581 wait complete. 2024-05-08T04:08:25.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.280+0000 7f5353fff640 1 -- 172.21.15.170:0/745347369 >> v2:172.21.15.187:6816/101007897 conn(0x7f5340076390 msgr2=0x7f5340078850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:25.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.280+0000 7f5353fff640 1 --2- 172.21.15.170:0/745347369 >> v2:172.21.15.187:6816/101007897 conn(0x7f5340076390 0x7f5340078850 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f53541f2420 tx=0x7f53480079b0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.280+0000 7f5353fff640 1 -- 172.21.15.170:0/745347369 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f535406fc80 msgr2=0x7f535407f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:25.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.280+0000 7f5353fff640 1 --2- 172.21.15.170:0/745347369 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f535406fc80 0x7f535407f190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f53540b14f0 tx=0x7f534c004a40 comp rx=0 tx=0).stop 2024-05-08T04:08:25.283 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.280+0000 7f5353fff640 1 -- 172.21.15.170:0/745347369 shutdown_connections 2024-05-08T04:08:25.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.280+0000 7f5353fff640 1 --2- 172.21.15.170:0/745347369 >> v2:172.21.15.187:6816/101007897 conn(0x7f5340076390 0x7f5340078850 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.280+0000 7f5353fff640 1 --2- 172.21.15.170:0/745347369 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f535414bd60 0x7f53541ee480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.280+0000 7f5353fff640 1 --2- 172.21.15.170:0/745347369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53540739c0 0x7f535407e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.280+0000 7f5353fff640 1 --2- 172.21.15.170:0/745347369 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f535406fc80 0x7f535407f190 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.280+0000 7f5353fff640 1 -- 172.21.15.170:0/745347369 >> 172.21.15.170:0/745347369 conn(0x7f53540b4bb0 msgr2=0x7f535414b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:25.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.280+0000 7f5353fff640 1 -- 172.21.15.170:0/745347369 shutdown_connections 2024-05-08T04:08:25.284 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.280+0000 7f5353fff640 1 -- 172.21.15.170:0/745347369 wait complete. 2024-05-08T04:08:25.301 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542147 for osd.0 2024-05-08T04:08:25.301 DEBUG:teuthology.parallel:result is None 2024-05-08T04:08:25.301 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542147 for osd.7 2024-05-08T04:08:25.302 DEBUG:teuthology.parallel:result is None 2024-05-08T04:08:25.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.320+0000 7f0b58ba8640 1 -- 172.21.15.170:0/1262601489 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f0b4c14f050 con 0x7f0b4c034540 2024-05-08T04:08:25.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.320+0000 7f0b3ffff640 1 -- 172.21.15.170:0/1262601489 <== mon.2 v2:172.21.15.170:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0b48068400 con 0x7f0b4c034540 2024-05-08T04:08:25.322 INFO:teuthology.orchestra.run.smithi170.stdout:60129542147 2024-05-08T04:08:25.324 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.322+0000 7f0b58ba8640 1 -- 172.21.15.170:0/1262601489 >> v2:172.21.15.187:6816/101007897 conn(0x7f0b380762e0 msgr2=0x7f0b380787a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:25.324 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.322+0000 7f0b58ba8640 1 --2- 172.21.15.170:0/1262601489 >> v2:172.21.15.187:6816/101007897 conn(0x7f0b380762e0 0x7f0b380787a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0b5406c270 tx=0x7f0b54068050 comp rx=0 tx=0).stop 2024-05-08T04:08:25.324 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.322+0000 7f0b58ba8640 1 -- 172.21.15.170:0/1262601489 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0b4c034540 msgr2=0x7f0b4c14eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:25.324 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.322+0000 7f0b58ba8640 1 --2- 172.21.15.170:0/1262601489 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0b4c034540 0x7f0b4c14eb10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0b4800c820 tx=0x7f0b4800cce0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.324 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.322+0000 7f0b58ba8640 1 -- 172.21.15.170:0/1262601489 shutdown_connections 2024-05-08T04:08:25.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.322+0000 7f0b58ba8640 1 --2- 172.21.15.170:0/1262601489 >> v2:172.21.15.187:6816/101007897 conn(0x7f0b380762e0 0x7f0b380787a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.322+0000 7f0b58ba8640 1 --2- 172.21.15.170:0/1262601489 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b4c02dcb0 0x7f0b4c164b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.322+0000 7f0b58ba8640 1 --2- 172.21.15.170:0/1262601489 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0b4c034540 0x7f0b4c14eb10 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.322+0000 7f0b58ba8640 1 --2- 172.21.15.170:0/1262601489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b4c02d1e0 0x7f0b4c02b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.322+0000 7f0b58ba8640 1 -- 172.21.15.170:0/1262601489 >> 172.21.15.170:0/1262601489 conn(0x7f0b4c036610 msgr2=0x7f0b4c0c3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:25.325 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.324+0000 7f0b58ba8640 1 -- 172.21.15.170:0/1262601489 shutdown_connections 2024-05-08T04:08:25.326 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.324+0000 7f0b58ba8640 1 -- 172.21.15.170:0/1262601489 wait complete. 2024-05-08T04:08:25.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.327+0000 7fd252117640 1 -- 172.21.15.170:0/1662155208 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fd24c06d6d0 con 0x7fd24c144f70 2024-05-08T04:08:25.329 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.327+0000 7fd233fff640 1 -- 172.21.15.170:0/1662155208 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd23c0615d0 con 0x7fd24c144f70 2024-05-08T04:08:25.329 INFO:teuthology.orchestra.run.smithi170.stdout:55834574852 2024-05-08T04:08:25.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.329+0000 7fd252117640 1 -- 172.21.15.170:0/1662155208 >> v2:172.21.15.187:6816/101007897 conn(0x7fd238076290 msgr2=0x7fd238078750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:25.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.329+0000 7fd252117640 1 --2- 172.21.15.170:0/1662155208 >> v2:172.21.15.187:6816/101007897 conn(0x7fd238076290 0x7fd238078750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd2400015f0 tx=0x7fd240007970 comp rx=0 tx=0).stop 2024-05-08T04:08:25.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.329+0000 7fd252117640 1 -- 172.21.15.170:0/1662155208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd24c144f70 msgr2=0x7fd24c140700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:25.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.329+0000 7fd252117640 1 --2- 172.21.15.170:0/1662155208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd24c144f70 0x7fd24c140700 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00d5c0 tx=0x7fd23c00da80 comp rx=0 tx=0).stop 2024-05-08T04:08:25.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.330+0000 7fd252117640 1 -- 172.21.15.170:0/1662155208 shutdown_connections 2024-05-08T04:08:25.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.330+0000 7fd252117640 1 --2- 172.21.15.170:0/1662155208 >> v2:172.21.15.187:6816/101007897 conn(0x7fd238076290 0x7fd238078750 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.330+0000 7fd252117640 1 --2- 172.21.15.170:0/1662155208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd24c144f70 0x7fd24c140700 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.330+0000 7fd252117640 1 --2- 172.21.15.170:0/1662155208 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd24c0b2450 0x7fd24c1401c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.330+0000 7fd252117640 1 --2- 172.21.15.170:0/1662155208 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd24c073f50 0x7fd24c0abbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.330+0000 7fd252117640 1 -- 172.21.15.170:0/1662155208 >> 172.21.15.170:0/1662155208 conn(0x7fd24c0b5fb0 msgr2=0x7fd24c0b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:25.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.330+0000 7fd252117640 1 -- 172.21.15.170:0/1662155208 shutdown_connections 2024-05-08T04:08:25.332 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.330+0000 7fd252117640 1 -- 172.21.15.170:0/1662155208 wait complete. 2024-05-08T04:08:25.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.337+0000 7f50938ed640 1 -- 172.21.15.170:0/2036052514 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f50840c1d90 con 0x7f508416a730 2024-05-08T04:08:25.339 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.338+0000 7f5081ffb640 1 -- 172.21.15.170:0/2036052514 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f507c063cb0 con 0x7f508416a730 2024-05-08T04:08:25.340 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542147 for osd.5 2024-05-08T04:08:25.340 INFO:teuthology.orchestra.run.smithi170.stdout:60129542147 2024-05-08T04:08:25.340 DEBUG:teuthology.parallel:result is None 2024-05-08T04:08:25.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.341+0000 7f50938ed640 1 -- 172.21.15.170:0/2036052514 >> v2:172.21.15.187:6816/101007897 conn(0x7f5070076360 msgr2=0x7f5070078820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:25.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.341+0000 7f50938ed640 1 --2- 172.21.15.170:0/2036052514 >> v2:172.21.15.187:6816/101007897 conn(0x7f5070076360 0x7f5070078820 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f508800a990 tx=0x7f5088008040 comp rx=0 tx=0).stop 2024-05-08T04:08:25.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.341+0000 7f50938ed640 1 -- 172.21.15.170:0/2036052514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f508416a730 msgr2=0x7f50840bfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:25.343 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.341+0000 7f50938ed640 1 --2- 172.21.15.170:0/2036052514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f508416a730 0x7f50840bfd50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f507c00c730 tx=0x7f507c00cbf0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.341+0000 7f50938ed640 1 -- 172.21.15.170:0/2036052514 shutdown_connections 2024-05-08T04:08:25.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.341+0000 7f50938ed640 1 --2- 172.21.15.170:0/2036052514 >> v2:172.21.15.187:6816/101007897 conn(0x7f5070076360 0x7f5070078820 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.341+0000 7f50938ed640 1 --2- 172.21.15.170:0/2036052514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f508402b600 0x7f508416a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.341+0000 7f50938ed640 1 --2- 172.21.15.170:0/2036052514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f508416a730 0x7f50840bfd50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.341+0000 7f50938ed640 1 --2- 172.21.15.170:0/2036052514 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5084037290 0x7f508402b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.341+0000 7f50938ed640 1 -- 172.21.15.170:0/2036052514 >> 172.21.15.170:0/2036052514 conn(0x7f508403a730 msgr2=0x7f5084032eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:25.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.341+0000 7f50938ed640 1 -- 172.21.15.170:0/2036052514 shutdown_connections 2024-05-08T04:08:25.344 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.341+0000 7f50938ed640 1 -- 172.21.15.170:0/2036052514 wait complete. 2024-05-08T04:08:25.346 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.2 2024-05-08T04:08:25.346 DEBUG:teuthology.parallel:result is None 2024-05-08T04:08:25.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.348+0000 7f3c137e9640 1 -- 172.21.15.170:0/99774960 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f3c0c137190 con 0x7f3c0c0b06c0 2024-05-08T04:08:25.351 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.349+0000 7f3c01ffb640 1 -- 172.21.15.170:0/99774960 <== mon.2 v2:172.21.15.170:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3bfc060ac0 con 0x7f3c0c0b06c0 2024-05-08T04:08:25.351 INFO:teuthology.orchestra.run.smithi170.stdout:60129542147 2024-05-08T04:08:25.354 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.352+0000 7f3c137e9640 1 -- 172.21.15.170:0/99774960 >> v2:172.21.15.187:6816/101007897 conn(0x7f3bf0076490 msgr2=0x7f3bf0078950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:25.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.352+0000 7f3c137e9640 1 --2- 172.21.15.170:0/99774960 >> v2:172.21.15.187:6816/101007897 conn(0x7f3bf0076490 0x7f3bf0078950 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c137440 tx=0x7f3c04007d30 comp rx=0 tx=0).stop 2024-05-08T04:08:25.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.352+0000 7f3c137e9640 1 -- 172.21.15.170:0/99774960 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3c0c0b06c0 msgr2=0x7f3c0c13a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:25.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.352+0000 7f3c137e9640 1 --2- 172.21.15.170:0/99774960 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3c0c0b06c0 0x7f3c0c13a7b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc00b410 tx=0x7f3bfc00b8d0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.352+0000 7f3c137e9640 1 -- 172.21.15.170:0/99774960 shutdown_connections 2024-05-08T04:08:25.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.352+0000 7f3c137e9640 1 --2- 172.21.15.170:0/99774960 >> v2:172.21.15.187:6816/101007897 conn(0x7f3bf0076490 0x7f3bf0078950 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.352+0000 7f3c137e9640 1 --2- 172.21.15.170:0/99774960 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3c0c143090 0x7f3c0c13f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.352+0000 7f3c137e9640 1 --2- 172.21.15.170:0/99774960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c0c139500 0x7f3c0c13f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.352+0000 7f3c137e9640 1 --2- 172.21.15.170:0/99774960 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3c0c0b06c0 0x7f3c0c13a7b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.352+0000 7f3c137e9640 1 -- 172.21.15.170:0/99774960 >> 172.21.15.170:0/99774960 conn(0x7f3c0c0b5250 msgr2=0x7f3c0c1434a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:25.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.352+0000 7f3c137e9640 1 -- 172.21.15.170:0/99774960 shutdown_connections 2024-05-08T04:08:25.355 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.352+0000 7f3c137e9640 1 -- 172.21.15.170:0/99774960 wait complete. 2024-05-08T04:08:25.358 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542147 for osd.4 2024-05-08T04:08:25.359 DEBUG:teuthology.parallel:result is None 2024-05-08T04:08:25.364 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.362+0000 7f3e5cdc8640 1 -- 172.21.15.170:0/4114720189 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f3e50035350 con 0x7f3e5002fe00 2024-05-08T04:08:25.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.362+0000 7f96db59e640 1 -- 172.21.15.170:0/225136705 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f96c0005370 con 0x7f96d4007190 2024-05-08T04:08:25.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.363+0000 7f3e43fff640 1 -- 172.21.15.170:0/4114720189 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3e580fd7d0 con 0x7f3e5002fe00 2024-05-08T04:08:25.365 INFO:teuthology.orchestra.run.smithi170.stdout:60129542147 2024-05-08T04:08:25.365 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.363+0000 7f96caffd640 1 -- 172.21.15.170:0/225136705 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f96cc082820 con 0x7f96d4007190 2024-05-08T04:08:25.366 INFO:teuthology.orchestra.run.smithi170.stdout:60129542147 2024-05-08T04:08:25.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.365+0000 7f3e5cdc8640 1 -- 172.21.15.170:0/4114720189 >> v2:172.21.15.187:6816/101007897 conn(0x7f3e3c0764c0 msgr2=0x7f3e3c078980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:25.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.365+0000 7f3e5cdc8640 1 --2- 172.21.15.170:0/4114720189 >> v2:172.21.15.187:6816/101007897 conn(0x7f3e3c0764c0 0x7f3e3c078980 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3e50031b40 tx=0x7f3e4c007950 comp rx=0 tx=0).stop 2024-05-08T04:08:25.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.365+0000 7f3e5cdc8640 1 -- 172.21.15.170:0/4114720189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e5002fe00 msgr2=0x7f3e500bc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:25.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.365+0000 7f3e5cdc8640 1 --2- 172.21.15.170:0/4114720189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e5002fe00 0x7f3e500bc3f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3e58065ed0 tx=0x7f3e58074cf0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.365+0000 7f3e5cdc8640 1 -- 172.21.15.170:0/4114720189 shutdown_connections 2024-05-08T04:08:25.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.365+0000 7f3e5cdc8640 1 --2- 172.21.15.170:0/4114720189 >> v2:172.21.15.187:6816/101007897 conn(0x7f3e3c0764c0 0x7f3e3c078980 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.365+0000 7f3e5cdc8640 1 --2- 172.21.15.170:0/4114720189 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3e50032400 0x7f3e5002a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.365+0000 7f3e5cdc8640 1 --2- 172.21.15.170:0/4114720189 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3e50161450 0x7f3e5002ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.365+0000 7f3e5cdc8640 1 --2- 172.21.15.170:0/4114720189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e5002fe00 0x7f3e500bc3f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.365+0000 7f3e5cdc8640 1 -- 172.21.15.170:0/4114720189 >> 172.21.15.170:0/4114720189 conn(0x7f3e50006ad0 msgr2=0x7f3e50005c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:25.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.365+0000 7f3e5cdc8640 1 -- 172.21.15.170:0/4114720189 shutdown_connections 2024-05-08T04:08:25.368 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.365+0000 7f3e5cdc8640 1 -- 172.21.15.170:0/4114720189 wait complete. 2024-05-08T04:08:25.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.366+0000 7f96db59e640 1 -- 172.21.15.170:0/225136705 >> v2:172.21.15.187:6816/101007897 conn(0x7f96bc076390 msgr2=0x7f96bc078850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:25.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.366+0000 7f96db59e640 1 --2- 172.21.15.170:0/225136705 >> v2:172.21.15.187:6816/101007897 conn(0x7f96bc076390 0x7f96bc078850 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f96dc074e40 tx=0x7f96dc06b300 comp rx=0 tx=0).stop 2024-05-08T04:08:25.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.366+0000 7f96db59e640 1 -- 172.21.15.170:0/225136705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96d4007190 msgr2=0x7f96d40a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:25.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.366+0000 7f96db59e640 1 --2- 172.21.15.170:0/225136705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96d4007190 0x7f96d40a3780 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f96cc033000 tx=0x7f96cc0046c0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.366+0000 7f96db59e640 1 -- 172.21.15.170:0/225136705 shutdown_connections 2024-05-08T04:08:25.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.366+0000 7f96db59e640 1 --2- 172.21.15.170:0/225136705 >> v2:172.21.15.187:6816/101007897 conn(0x7f96bc076390 0x7f96bc078850 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.366+0000 7f96db59e640 1 --2- 172.21.15.170:0/225136705 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f96d40338f0 0x7f96d4151d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.366+0000 7f96db59e640 1 --2- 172.21.15.170:0/225136705 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f96d40097a0 0x7f96d40a5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.366+0000 7f96db59e640 1 --2- 172.21.15.170:0/225136705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96d4007190 0x7f96d40a3780 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.366+0000 7f96db59e640 1 -- 172.21.15.170:0/225136705 >> 172.21.15.170:0/225136705 conn(0x7f96d40061f0 msgr2=0x7f96d41574b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:25.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.366+0000 7f96db59e640 1 -- 172.21.15.170:0/225136705 shutdown_connections 2024-05-08T04:08:25.370 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.366+0000 7f96db59e640 1 -- 172.21.15.170:0/225136705 wait complete. 2024-05-08T04:08:25.370 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542147 for osd.1 2024-05-08T04:08:25.371 DEBUG:teuthology.parallel:result is None 2024-05-08T04:08:25.380 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542147 for osd.3 2024-05-08T04:08:25.380 DEBUG:teuthology.parallel:result is None 2024-05-08T04:08:25.380 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542147 for osd.6 2024-05-08T04:08:25.380 DEBUG:teuthology.parallel:result is None 2024-05-08T04:08:25.380 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T04:08:25.381 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:08:25.518 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.516+0000 7f461b22a640 1 Processor -- start 2024-05-08T04:08:25.519 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.516+0000 7f461b22a640 1 -- start start 2024-05-08T04:08:25.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.517+0000 7f461b22a640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f46140b1980 0x7f461413e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:25.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.517+0000 7f461b22a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f46140728a0 0x7f461413eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:25.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.517+0000 7f461b22a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4614074150 con 0x7f46140724c0 2024-05-08T04:08:25.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.517+0000 7f461b22a640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7f4614144cb0 con 0x7f46140b1980 2024-05-08T04:08:25.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.517+0000 7f461b22a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f4614074430 con 0x7f46140728a0 2024-05-08T04:08:25.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.517+0000 7f4613fff640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f46140b1980 0x7f461413e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:25.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.517+0000 7f4613fff640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f46140b1980 0x7f461413e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41052/0 (socket says 172.21.15.170:41052) 2024-05-08T04:08:25.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.517+0000 7f4613fff640 1 -- 172.21.15.170:0/837007215 learned_addr learned my addr 172.21.15.170:0/837007215 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:25.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.517+0000 7f46197c7640 1 --2- 172.21.15.170:0/837007215 >> v2:172.21.15.187:3300/0 conn(0x7f46140728a0 0x7f461413eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:25.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.517+0000 7f4613fff640 1 -- 172.21.15.170:0/837007215 >> v2:172.21.15.187:3300/0 conn(0x7f46140728a0 msgr2=0x7f461413eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:25.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.517+0000 7f4613fff640 1 --2- 172.21.15.170:0/837007215 >> v2:172.21.15.187:3300/0 conn(0x7f46140728a0 0x7f461413eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.518+0000 7f4613fff640 1 -- 172.21.15.170:0/837007215 >> v1:172.21.15.187:6789/0 conn(0x7f46140724c0 legacy=0x7f4614145140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:25.520 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.518+0000 7f4613fff640 1 -- 172.21.15.170:0/837007215 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46140aa650 con 0x7f46140b1980 2024-05-08T04:08:25.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.518+0000 7f4613fff640 1 --2- 172.21.15.170:0/837007215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46140b1980 0x7f461413e860 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4614071920 tx=0x7f460802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b5a7743d3307f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:25.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.518+0000 7f46137fe640 1 -- 172.21.15.170:0/837007215 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460802c9f0 con 0x7f46140b1980 2024-05-08T04:08:25.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.518+0000 7f46137fe640 1 -- 172.21.15.170:0/837007215 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f460802cb90 con 0x7f46140b1980 2024-05-08T04:08:25.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.518+0000 7f46137fe640 1 -- 172.21.15.170:0/837007215 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46080343f0 con 0x7f46140b1980 2024-05-08T04:08:25.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.519+0000 7f461b22a640 1 -- 172.21.15.170:0/837007215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46140b1980 msgr2=0x7f461413e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:25.522 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.519+0000 7f461b22a640 1 --2- 172.21.15.170:0/837007215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46140b1980 0x7f461413e860 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4614071920 tx=0x7f460802be50 comp rx=0 tx=0).stop 2024-05-08T04:08:25.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.519+0000 7f461b22a640 1 -- 172.21.15.170:0/837007215 shutdown_connections 2024-05-08T04:08:25.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.519+0000 7f461b22a640 1 --2- 172.21.15.170:0/837007215 >> v2:172.21.15.187:3300/0 conn(0x7f46140728a0 0x7f461413eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.519+0000 7f461b22a640 1 --2- 172.21.15.170:0/837007215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46140b1980 0x7f461413e860 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.519+0000 7f461b22a640 1 -- 172.21.15.170:0/837007215 >> 172.21.15.170:0/837007215 conn(0x7f46140b7110 msgr2=0x7f46140b7520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:25.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.519+0000 7f461b22a640 1 -- 172.21.15.170:0/837007215 shutdown_connections 2024-05-08T04:08:25.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.519+0000 7f461b22a640 1 -- 172.21.15.170:0/837007215 wait complete. 2024-05-08T04:08:25.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.520+0000 7f461b22a640 1 Processor -- start 2024-05-08T04:08:25.523 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.520+0000 7f461b22a640 1 -- start start 2024-05-08T04:08:25.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.521+0000 7f461b22a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46140724c0 0x7f4614143bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:25.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.521+0000 7f461b22a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46140728a0 0x7f46141d6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:25.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.521+0000 7f461b22a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46140b1980 0x7f46141d6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:25.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.521+0000 7f461b22a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f461414c000 con 0x7f46140728a0 2024-05-08T04:08:25.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.521+0000 7f461b22a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f461414be80 con 0x7f46140b1980 2024-05-08T04:08:25.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.521+0000 7f461b22a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f461414c180 con 0x7f46140724c0 2024-05-08T04:08:25.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.521+0000 7f4613fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46140728a0 0x7f46141d6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:25.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.521+0000 7f4618fc6640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46140724c0 0x7f4614143bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:25.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.521+0000 7f4613fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46140728a0 0x7f46141d6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41058/0 (socket says 172.21.15.170:41058) 2024-05-08T04:08:25.526 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.521+0000 7f4613fff640 1 -- 172.21.15.170:0/1699020219 learned_addr learned my addr 172.21.15.170:0/1699020219 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:25.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.521+0000 7f46197c7640 1 --2- 172.21.15.170:0/1699020219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46140b1980 0x7f46141d6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:25.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.521+0000 7f4613fff640 1 -- 172.21.15.170:0/1699020219 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46140724c0 msgr2=0x7f4614143bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:25.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.521+0000 7f4613fff640 1 --2- 172.21.15.170:0/1699020219 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46140724c0 0x7f4614143bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.521+0000 7f4613fff640 1 -- 172.21.15.170:0/1699020219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46140b1980 msgr2=0x7f46141d6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:25.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.521+0000 7f4613fff640 1 --2- 172.21.15.170:0/1699020219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46140b1980 0x7f46141d6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.521+0000 7f4613fff640 1 -- 172.21.15.170:0/1699020219 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4614138770 con 0x7f46140728a0 2024-05-08T04:08:25.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.521+0000 7f4613fff640 1 --2- 172.21.15.170:0/1699020219 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46140728a0 0x7f46141d6710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4608036040 tx=0x7f4608004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:25.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.521+0000 7f46117fa640 1 -- 172.21.15.170:0/1699020219 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4608044070 con 0x7f46140728a0 2024-05-08T04:08:25.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.521+0000 7f46117fa640 1 -- 172.21.15.170:0/1699020219 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f460803e430 con 0x7f46140728a0 2024-05-08T04:08:25.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.521+0000 7f461b22a640 1 -- 172.21.15.170:0/1699020219 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46141372d0 con 0x7f46140728a0 2024-05-08T04:08:25.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.521+0000 7f46117fa640 1 -- 172.21.15.170:0/1699020219 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4608009510 con 0x7f46140728a0 2024-05-08T04:08:25.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.522+0000 7f461b22a640 1 -- 172.21.15.170:0/1699020219 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46141d7190 con 0x7f46140728a0 2024-05-08T04:08:25.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.523+0000 7f461b22a640 1 -- 172.21.15.170:0/1699020219 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4614069bd0 con 0x7f46140728a0 2024-05-08T04:08:25.527 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.523+0000 7f46117fa640 1 -- 172.21.15.170:0/1699020219 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f4608005d10 con 0x7f46140728a0 2024-05-08T04:08:25.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.523+0000 7f46117fa640 1 --2- 172.21.15.170:0/1699020219 >> v2:172.21.15.187:6816/101007897 conn(0x7f4600076290 0x7f4600078750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:25.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.523+0000 7f46117fa640 1 -- 172.21.15.170:0/1699020219 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4090+0+0 (secure 0 0 0) 0x7f46080b96a0 con 0x7f46140728a0 2024-05-08T04:08:25.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.524+0000 7f4618fc6640 1 --2- 172.21.15.170:0/1699020219 >> v2:172.21.15.187:6816/101007897 conn(0x7f4600076290 0x7f4600078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:25.528 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.524+0000 7f4618fc6640 1 --2- 172.21.15.170:0/1699020219 >> v2:172.21.15.187:6816/101007897 conn(0x7f4600076290 0x7f4600078750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f46040017c0 tx=0x7f4604009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:25.529 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.528+0000 7f46117fa640 1 -- 172.21.15.170:0/1699020219 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f460804e030 con 0x7f46140728a0 2024-05-08T04:08:25.710 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.708+0000 7f461b22a640 1 -- 172.21.15.170:0/1699020219 --> v2:172.21.15.187:6816/101007897 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f46141374a0 con 0x7f4600076290 2024-05-08T04:08:25.711 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.709+0000 7f46117fa640 1 -- 172.21.15.170:0/1699020219 <== mgr.4107 v2:172.21.15.187:6816/101007897 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+35551 (secure 0 0 0) 0x7f46141374a0 con 0x7f4600076290 2024-05-08T04:08:25.712 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:08:25.712 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-05-08T04:08:25.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.712+0000 7f461b22a640 1 -- 172.21.15.170:0/1699020219 >> v2:172.21.15.187:6816/101007897 conn(0x7f4600076290 msgr2=0x7f4600078750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:25.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.712+0000 7f461b22a640 1 --2- 172.21.15.170:0/1699020219 >> v2:172.21.15.187:6816/101007897 conn(0x7f4600076290 0x7f4600078750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f46040017c0 tx=0x7f4604009040 comp rx=0 tx=0).stop 2024-05-08T04:08:25.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.713+0000 7f461b22a640 1 -- 172.21.15.170:0/1699020219 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46140728a0 msgr2=0x7f46141d6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:25.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.713+0000 7f461b22a640 1 --2- 172.21.15.170:0/1699020219 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46140728a0 0x7f46141d6710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4608036040 tx=0x7f4608004a40 comp rx=0 tx=0).stop 2024-05-08T04:08:25.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.713+0000 7f461b22a640 1 -- 172.21.15.170:0/1699020219 shutdown_connections 2024-05-08T04:08:25.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.713+0000 7f461b22a640 1 --2- 172.21.15.170:0/1699020219 >> v2:172.21.15.187:6816/101007897 conn(0x7f4600076290 0x7f4600078750 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.713+0000 7f461b22a640 1 --2- 172.21.15.170:0/1699020219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46140b1980 0x7f46141d6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.713+0000 7f461b22a640 1 --2- 172.21.15.170:0/1699020219 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46140728a0 0x7f46141d6710 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.715 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.713+0000 7f461b22a640 1 --2- 172.21.15.170:0/1699020219 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f46140724c0 0x7f4614143bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.713+0000 7f461b22a640 1 -- 172.21.15.170:0/1699020219 >> 172.21.15.170:0/1699020219 conn(0x7f46140b7110 msgr2=0x7f4614135ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:25.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.713+0000 7f461b22a640 1 -- 172.21.15.170:0/1699020219 shutdown_connections 2024-05-08T04:08:25.716 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.713+0000 7f461b22a640 1 -- 172.21.15.170:0/1699020219 wait complete. 2024-05-08T04:08:25.726 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2024-05-08T04:08:25.655549+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":221668,"kb_used_data":1304,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754753052,"statfs":{"total":773094113280,"available":772867125248,"internally_reserved":0,"allocated":1335296,"data_stored":1440672,"data_compressed":10222,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000349"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T04:08:22.695943+0000","last_change":"2024-05-08T04:08:21.697423+0000","last_active":"2024-05-08T04:08:22.695943+0000","last_peered":"2024-05-08T04:08:22.695943+0000","last_clean":"2024-05-08T04:08:22.695943+0000","last_became_active":"2024-05-08T04:08:21.697149+0000","last_became_peered":"2024-05-08T04:08:21.697149+0000","last_unstale":"2024-05-08T04:08:22.695943+0000","last_undegraded":"2024-05-08T04:08:22.695943+0000","last_fullsized":"2024-05-08T04:08:22.695943+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:59:38.354372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T04:08:23.214959+0000","last_change":"2024-05-08T04:08:21.696942+0000","last_active":"2024-05-08T04:08:23.214959+0000","last_peered":"2024-05-08T04:08:23.214959+0000","last_clean":"2024-05-08T04:08:23.214959+0000","last_became_active":"2024-05-08T04:08:21.696632+0000","last_became_peered":"2024-05-08T04:08:21.696632+0000","last_unstale":"2024-05-08T04:08:23.214959+0000","last_undegraded":"2024-05-08T04:08:23.214959+0000","last_fullsized":"2024-05-08T04:08:23.214959+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:23:25.156383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T04:08:23.857828+0000","last_change":"2024-05-08T04:08:21.696814+0000","last_active":"2024-05-08T04:08:23.857828+0000","last_peered":"2024-05-08T04:08:23.857828+0000","last_clean":"2024-05-08T04:08:23.857828+0000","last_became_active":"2024-05-08T04:08:21.696403+0000","last_became_peered":"2024-05-08T04:08:21.696403+0000","last_unstale":"2024-05-08T04:08:23.857828+0000","last_undegraded":"2024-05-08T04:08:23.857828+0000","last_fullsized":"2024-05-08T04:08:23.857828+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:02:04.075137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T04:08:23.215112+0000","last_change":"2024-05-08T04:08:21.693307+0000","last_active":"2024-05-08T04:08:23.215112+0000","last_peered":"2024-05-08T04:08:23.215112+0000","last_clean":"2024-05-08T04:08:23.215112+0000","last_became_active":"2024-05-08T04:08:21.693014+0000","last_became_peered":"2024-05-08T04:08:21.693014+0000","last_unstale":"2024-05-08T04:08:23.215112+0000","last_undegraded":"2024-05-08T04:08:23.215112+0000","last_fullsized":"2024-05-08T04:08:23.215112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:56.218680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T04:08:22.696064+0000","last_change":"2024-05-08T04:08:21.699221+0000","last_active":"2024-05-08T04:08:22.696064+0000","last_peered":"2024-05-08T04:08:22.696064+0000","last_clean":"2024-05-08T04:08:22.696064+0000","last_became_active":"2024-05-08T04:08:21.699013+0000","last_became_peered":"2024-05-08T04:08:21.699013+0000","last_unstale":"2024-05-08T04:08:22.696064+0000","last_undegraded":"2024-05-08T04:08:22.696064+0000","last_fullsized":"2024-05-08T04:08:22.696064+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:23:08.040183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T04:08:22.702027+0000","last_change":"2024-05-08T04:08:21.799283+0000","last_active":"2024-05-08T04:08:22.702027+0000","last_peered":"2024-05-08T04:08:22.702027+0000","last_clean":"2024-05-08T04:08:22.702027+0000","last_became_active":"2024-05-08T04:08:21.798995+0000","last_became_peered":"2024-05-08T04:08:21.798995+0000","last_unstale":"2024-05-08T04:08:22.702027+0000","last_undegraded":"2024-05-08T04:08:22.702027+0000","last_fullsized":"2024-05-08T04:08:22.702027+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:37:16.795888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T04:08:23.857827+0000","last_change":"2024-05-08T04:08:21.697070+0000","last_active":"2024-05-08T04:08:23.857827+0000","last_peered":"2024-05-08T04:08:23.857827+0000","last_clean":"2024-05-08T04:08:23.857827+0000","last_became_active":"2024-05-08T04:08:21.696370+0000","last_became_peered":"2024-05-08T04:08:21.696370+0000","last_unstale":"2024-05-08T04:08:23.857827+0000","last_undegraded":"2024-05-08T04:08:23.857827+0000","last_fullsized":"2024-05-08T04:08:23.857827+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:54.981822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T04:08:22.695951+0000","last_change":"2024-05-08T04:08:21.696942+0000","last_active":"2024-05-08T04:08:22.695951+0000","last_peered":"2024-05-08T04:08:22.695951+0000","last_clean":"2024-05-08T04:08:22.695951+0000","last_became_active":"2024-05-08T04:08:21.696659+0000","last_became_peered":"2024-05-08T04:08:21.696659+0000","last_unstale":"2024-05-08T04:08:22.695951+0000","last_undegraded":"2024-05-08T04:08:22.695951+0000","last_fullsized":"2024-05-08T04:08:22.695951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:50:44.209520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":104,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T04:08:23.857745+0000","last_change":"2024-05-08T04:08:20.691470+0000","last_active":"2024-05-08T04:08:23.857745+0000","last_peered":"2024-05-08T04:08:23.857745+0000","last_clean":"2024-05-08T04:08:23.857745+0000","last_became_active":"2024-05-08T04:08:20.691204+0000","last_became_peered":"2024-05-08T04:08:20.691204+0000","last_unstale":"2024-05-08T04:08:23.857745+0000","last_undegraded":"2024-05-08T04:08:23.857745+0000","last_fullsized":"2024-05-08T04:08:23.857745+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:19.679313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:19.679313+0000","last_clean_scrub_stamp":"2024-05-08T04:08:19.679313+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:52:31.185659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":10222,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":577937,"data_compressed":5111,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":49009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":49009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":147456,"data_stored":44381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":155648,"data_stored":49009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":49009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":577937,"data_compressed":5111,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":44381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5111,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5111,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:08:25.727 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:08:25.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.824+0000 7fd4e63b8640 1 Processor -- start 2024-05-08T04:08:25.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.824+0000 7fd4e63b8640 1 -- start start 2024-05-08T04:08:25.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.824+0000 7fd4e63b8640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd4e00b43a0 0x7fd4e00afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:25.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.824+0000 7fd4e63b8640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fd4e0140e60 0x7fd4e014e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:25.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.824+0000 7fd4e63b8640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4e00b0310 con 0x7fd4e006dd60 2024-05-08T04:08:25.827 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.824+0000 7fd4e63b8640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7fd4e00af3f0 con 0x7fd4e0140e60 2024-05-08T04:08:25.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.824+0000 7fd4e63b8640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd4e00af6d0 con 0x7fd4e00b43a0 2024-05-08T04:08:25.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.824+0000 7fd4e4955640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fd4e0140e60 0x7fd4e014e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:25.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.825+0000 7fd4df7fe640 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fd4e006dd60 0x7fd4e0140750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:38966/0 (socket says 172.21.15.170:38966) 2024-05-08T04:08:25.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.825+0000 7fd4df7fe640 1 -- 172.21.15.170:0/1333109318 learned_addr learned my addr 172.21.15.170:0/1333109318 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-05-08T04:08:25.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.825+0000 7fd4dffff640 1 --2- 172.21.15.170:0/1333109318 >> v2:172.21.15.187:3300/0 conn(0x7fd4e00b43a0 0x7fd4e00afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:25.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.825+0000 7fd4e4955640 1 -- 172.21.15.170:0/1333109318 >> v2:172.21.15.187:3300/0 conn(0x7fd4e00b43a0 msgr2=0x7fd4e00afdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:25.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.825+0000 7fd4e4955640 1 --2- 172.21.15.170:0/1333109318 >> v2:172.21.15.187:3300/0 conn(0x7fd4e00b43a0 0x7fd4e00afdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.825+0000 7fd4e4955640 1 -- 172.21.15.170:0/1333109318 >> v1:172.21.15.170:6789/0 conn(0x7fd4e006dd60 legacy=0x7fd4e0140750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:25.828 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.825+0000 7fd4e4955640 1 -- 172.21.15.170:0/1333109318 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4e00af9b0 con 0x7fd4e0140e60 2024-05-08T04:08:25.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.825+0000 7fd4e4955640 1 --2- 172.21.15.170:0/1333109318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4e0140e60 0x7fd4e014e0a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd4e006cf30 tx=0x7fd4d802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f949d01dd4d95484 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:25.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.826+0000 7fd4deffd640 1 -- 172.21.15.170:0/1333109318 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d8039070 con 0x7fd4e0140e60 2024-05-08T04:08:25.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.826+0000 7fd4deffd640 1 -- 172.21.15.170:0/1333109318 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4d802cc80 con 0x7fd4e0140e60 2024-05-08T04:08:25.831 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.826+0000 7fd4deffd640 1 -- 172.21.15.170:0/1333109318 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d8034660 con 0x7fd4e0140e60 2024-05-08T04:08:25.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.826+0000 7fd4e63b8640 1 -- 172.21.15.170:0/1333109318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4e0140e60 msgr2=0x7fd4e014e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:25.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.826+0000 7fd4e63b8640 1 --2- 172.21.15.170:0/1333109318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4e0140e60 0x7fd4e014e0a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd4e006cf30 tx=0x7fd4d802bed0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.826+0000 7fd4e63b8640 1 -- 172.21.15.170:0/1333109318 shutdown_connections 2024-05-08T04:08:25.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.826+0000 7fd4e63b8640 1 --2- 172.21.15.170:0/1333109318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4e0140e60 0x7fd4e014e0a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.826+0000 7fd4e63b8640 1 --2- 172.21.15.170:0/1333109318 >> v2:172.21.15.187:3300/0 conn(0x7fd4e00b43a0 0x7fd4e00afdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.826+0000 7fd4e63b8640 1 -- 172.21.15.170:0/1333109318 >> 172.21.15.170:0/1333109318 conn(0x7fd4e00b74f0 msgr2=0x7fd4e00b39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:25.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.826+0000 7fd4e63b8640 1 -- 172.21.15.170:0/1333109318 shutdown_connections 2024-05-08T04:08:25.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.826+0000 7fd4e63b8640 1 -- 172.21.15.170:0/1333109318 wait complete. 2024-05-08T04:08:25.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.827+0000 7fd4e63b8640 1 Processor -- start 2024-05-08T04:08:25.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.827+0000 7fd4e63b8640 1 -- start start 2024-05-08T04:08:25.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.828+0000 7fd4e63b8640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd4e006dd60 0x7fd4e00b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:25.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.828+0000 7fd4e63b8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4e00b43a0 0x7fd4e01d9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:25.832 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.828+0000 7fd4e63b8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4e0140e60 0x7fd4e00732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:25.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.828+0000 7fd4e63b8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e00b1600 con 0x7fd4e0140e60 2024-05-08T04:08:25.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.828+0000 7fd4e63b8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e00b1480 con 0x7fd4e00b43a0 2024-05-08T04:08:25.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.828+0000 7fd4e63b8640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fd4e006e1f0 con 0x7fd4e006dd60 2024-05-08T04:08:25.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.828+0000 7fd4e4955640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4e0140e60 0x7fd4e00732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:25.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.828+0000 7fd4dffff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd4e006dd60 0x7fd4e00b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:25.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.828+0000 7fd4e4955640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4e0140e60 0x7fd4e00732d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41082/0 (socket says 172.21.15.170:41082) 2024-05-08T04:08:25.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.828+0000 7fd4dffff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd4e006dd60 0x7fd4e00b1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:57776/0 (socket says 172.21.15.170:57776) 2024-05-08T04:08:25.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.828+0000 7fd4e4955640 1 -- 172.21.15.170:0/3085788980 learned_addr learned my addr 172.21.15.170:0/3085788980 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:25.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.828+0000 7fd4df7fe640 1 --2- 172.21.15.170:0/3085788980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4e00b43a0 0x7fd4e01d9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:25.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.828+0000 7fd4e4955640 1 -- 172.21.15.170:0/3085788980 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd4e006dd60 msgr2=0x7fd4e00b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:25.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.828+0000 7fd4e4955640 1 --2- 172.21.15.170:0/3085788980 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd4e006dd60 0x7fd4e00b1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.828+0000 7fd4e4955640 1 -- 172.21.15.170:0/3085788980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4e00b43a0 msgr2=0x7fd4e01d9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:25.833 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.828+0000 7fd4e4955640 1 --2- 172.21.15.170:0/3085788980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4e00b43a0 0x7fd4e01d9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:25.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.828+0000 7fd4e4955640 1 -- 172.21.15.170:0/3085788980 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4e00b22f0 con 0x7fd4e0140e60 2024-05-08T04:08:25.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.828+0000 7fd4e4955640 1 --2- 172.21.15.170:0/3085788980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4e0140e60 0x7fd4e00732d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8009e80 tx=0x7fd4d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:25.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.829+0000 7fd4dcff9640 1 -- 172.21.15.170:0/3085788980 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d8039050 con 0x7fd4e0140e60 2024-05-08T04:08:25.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.829+0000 7fd4dcff9640 1 -- 172.21.15.170:0/3085788980 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4d8045070 con 0x7fd4e0140e60 2024-05-08T04:08:25.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.829+0000 7fd4e63b8640 1 -- 172.21.15.170:0/3085788980 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4e01da000 con 0x7fd4e0140e60 2024-05-08T04:08:25.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.829+0000 7fd4dcff9640 1 -- 172.21.15.170:0/3085788980 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d803e830 con 0x7fd4e0140e60 2024-05-08T04:08:25.834 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.829+0000 7fd4e63b8640 1 -- 172.21.15.170:0/3085788980 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4e00726e0 con 0x7fd4e0140e60 2024-05-08T04:08:25.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.830+0000 7fd4e63b8640 1 -- 172.21.15.170:0/3085788980 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4e006b400 con 0x7fd4e0140e60 2024-05-08T04:08:25.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.830+0000 7fd4dcff9640 1 -- 172.21.15.170:0/3085788980 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7fd4d803e9d0 con 0x7fd4e0140e60 2024-05-08T04:08:25.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.831+0000 7fd4dcff9640 1 --2- 172.21.15.170:0/3085788980 >> v2:172.21.15.187:6816/101007897 conn(0x7fd4cc076290 0x7fd4cc078750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:25.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.831+0000 7fd4dcff9640 1 -- 172.21.15.170:0/3085788980 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4090+0+0 (secure 0 0 0) 0x7fd4d80b9af0 con 0x7fd4e0140e60 2024-05-08T04:08:25.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.831+0000 7fd4dffff640 1 --2- 172.21.15.170:0/3085788980 >> v2:172.21.15.187:6816/101007897 conn(0x7fd4cc076290 0x7fd4cc078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:25.835 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.831+0000 7fd4dffff640 1 --2- 172.21.15.170:0/3085788980 >> v2:172.21.15.187:6816/101007897 conn(0x7fd4cc076290 0x7fd4cc078750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0006fd0 tx=0x7fd4d0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:25.837 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:25.835+0000 7fd4dcff9640 1 -- 172.21.15.170:0/3085788980 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd4d80834e0 con 0x7fd4e0140e60 2024-05-08T04:08:26.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.012+0000 7fd4e63b8640 1 -- 172.21.15.170:0/3085788980 --> v2:172.21.15.187:6816/101007897 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd4e01d93f0 con 0x7fd4cc076290 2024-05-08T04:08:26.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.014+0000 7fd4dcff9640 1 -- 172.21.15.170:0/3085788980 <== mgr.4107 v2:172.21.15.187:6816/101007897 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+35551 (secure 0 0 0) 0x7fd4e01d93f0 con 0x7fd4cc076290 2024-05-08T04:08:26.016 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:08:26.017 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-05-08T04:08:26.018 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.016+0000 7fd4e63b8640 1 -- 172.21.15.170:0/3085788980 >> v2:172.21.15.187:6816/101007897 conn(0x7fd4cc076290 msgr2=0x7fd4cc078750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:26.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.016+0000 7fd4e63b8640 1 --2- 172.21.15.170:0/3085788980 >> v2:172.21.15.187:6816/101007897 conn(0x7fd4cc076290 0x7fd4cc078750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0006fd0 tx=0x7fd4d0008040 comp rx=0 tx=0).stop 2024-05-08T04:08:26.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.016+0000 7fd4e63b8640 1 -- 172.21.15.170:0/3085788980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4e0140e60 msgr2=0x7fd4e00732d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:26.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.016+0000 7fd4e63b8640 1 --2- 172.21.15.170:0/3085788980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4e0140e60 0x7fd4e00732d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8009e80 tx=0x7fd4d8004a40 comp rx=0 tx=0).stop 2024-05-08T04:08:26.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.016+0000 7fd4e63b8640 1 -- 172.21.15.170:0/3085788980 shutdown_connections 2024-05-08T04:08:26.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.016+0000 7fd4e63b8640 1 --2- 172.21.15.170:0/3085788980 >> v2:172.21.15.187:6816/101007897 conn(0x7fd4cc076290 0x7fd4cc078750 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:26.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.016+0000 7fd4e63b8640 1 --2- 172.21.15.170:0/3085788980 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd4e00b43a0 0x7fd4e01d9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:26.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.016+0000 7fd4e63b8640 1 --2- 172.21.15.170:0/3085788980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4e0140e60 0x7fd4e00732d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:26.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.016+0000 7fd4e63b8640 1 --2- 172.21.15.170:0/3085788980 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd4e006dd60 0x7fd4e00b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:26.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.016+0000 7fd4e63b8640 1 -- 172.21.15.170:0/3085788980 >> 172.21.15.170:0/3085788980 conn(0x7fd4e00b74f0 msgr2=0x7fd4e0139640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:26.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.016+0000 7fd4e63b8640 1 -- 172.21.15.170:0/3085788980 shutdown_connections 2024-05-08T04:08:26.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.016+0000 7fd4e63b8640 1 -- 172.21.15.170:0/3085788980 wait complete. 2024-05-08T04:08:26.029 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2024-05-08T04:08:25.655549+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":221668,"kb_used_data":1304,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754753052,"statfs":{"total":773094113280,"available":772867125248,"internally_reserved":0,"allocated":1335296,"data_stored":1440672,"data_compressed":10222,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000349"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T04:08:22.695943+0000","last_change":"2024-05-08T04:08:21.697423+0000","last_active":"2024-05-08T04:08:22.695943+0000","last_peered":"2024-05-08T04:08:22.695943+0000","last_clean":"2024-05-08T04:08:22.695943+0000","last_became_active":"2024-05-08T04:08:21.697149+0000","last_became_peered":"2024-05-08T04:08:21.697149+0000","last_unstale":"2024-05-08T04:08:22.695943+0000","last_undegraded":"2024-05-08T04:08:22.695943+0000","last_fullsized":"2024-05-08T04:08:22.695943+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:59:38.354372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T04:08:23.214959+0000","last_change":"2024-05-08T04:08:21.696942+0000","last_active":"2024-05-08T04:08:23.214959+0000","last_peered":"2024-05-08T04:08:23.214959+0000","last_clean":"2024-05-08T04:08:23.214959+0000","last_became_active":"2024-05-08T04:08:21.696632+0000","last_became_peered":"2024-05-08T04:08:21.696632+0000","last_unstale":"2024-05-08T04:08:23.214959+0000","last_undegraded":"2024-05-08T04:08:23.214959+0000","last_fullsized":"2024-05-08T04:08:23.214959+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:23:25.156383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T04:08:23.857828+0000","last_change":"2024-05-08T04:08:21.696814+0000","last_active":"2024-05-08T04:08:23.857828+0000","last_peered":"2024-05-08T04:08:23.857828+0000","last_clean":"2024-05-08T04:08:23.857828+0000","last_became_active":"2024-05-08T04:08:21.696403+0000","last_became_peered":"2024-05-08T04:08:21.696403+0000","last_unstale":"2024-05-08T04:08:23.857828+0000","last_undegraded":"2024-05-08T04:08:23.857828+0000","last_fullsized":"2024-05-08T04:08:23.857828+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:02:04.075137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T04:08:23.215112+0000","last_change":"2024-05-08T04:08:21.693307+0000","last_active":"2024-05-08T04:08:23.215112+0000","last_peered":"2024-05-08T04:08:23.215112+0000","last_clean":"2024-05-08T04:08:23.215112+0000","last_became_active":"2024-05-08T04:08:21.693014+0000","last_became_peered":"2024-05-08T04:08:21.693014+0000","last_unstale":"2024-05-08T04:08:23.215112+0000","last_undegraded":"2024-05-08T04:08:23.215112+0000","last_fullsized":"2024-05-08T04:08:23.215112+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:56.218680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T04:08:22.696064+0000","last_change":"2024-05-08T04:08:21.699221+0000","last_active":"2024-05-08T04:08:22.696064+0000","last_peered":"2024-05-08T04:08:22.696064+0000","last_clean":"2024-05-08T04:08:22.696064+0000","last_became_active":"2024-05-08T04:08:21.699013+0000","last_became_peered":"2024-05-08T04:08:21.699013+0000","last_unstale":"2024-05-08T04:08:22.696064+0000","last_undegraded":"2024-05-08T04:08:22.696064+0000","last_fullsized":"2024-05-08T04:08:22.696064+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:23:08.040183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T04:08:22.702027+0000","last_change":"2024-05-08T04:08:21.799283+0000","last_active":"2024-05-08T04:08:22.702027+0000","last_peered":"2024-05-08T04:08:22.702027+0000","last_clean":"2024-05-08T04:08:22.702027+0000","last_became_active":"2024-05-08T04:08:21.798995+0000","last_became_peered":"2024-05-08T04:08:21.798995+0000","last_unstale":"2024-05-08T04:08:22.702027+0000","last_undegraded":"2024-05-08T04:08:22.702027+0000","last_fullsized":"2024-05-08T04:08:22.702027+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:37:16.795888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T04:08:23.857827+0000","last_change":"2024-05-08T04:08:21.697070+0000","last_active":"2024-05-08T04:08:23.857827+0000","last_peered":"2024-05-08T04:08:23.857827+0000","last_clean":"2024-05-08T04:08:23.857827+0000","last_became_active":"2024-05-08T04:08:21.696370+0000","last_became_peered":"2024-05-08T04:08:21.696370+0000","last_unstale":"2024-05-08T04:08:23.857827+0000","last_undegraded":"2024-05-08T04:08:23.857827+0000","last_fullsized":"2024-05-08T04:08:23.857827+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:54.981822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T04:08:22.695951+0000","last_change":"2024-05-08T04:08:21.696942+0000","last_active":"2024-05-08T04:08:22.695951+0000","last_peered":"2024-05-08T04:08:22.695951+0000","last_clean":"2024-05-08T04:08:22.695951+0000","last_became_active":"2024-05-08T04:08:21.696659+0000","last_became_peered":"2024-05-08T04:08:21.696659+0000","last_unstale":"2024-05-08T04:08:22.695951+0000","last_undegraded":"2024-05-08T04:08:22.695951+0000","last_fullsized":"2024-05-08T04:08:22.695951+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:50:44.209520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":104,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T04:08:23.857745+0000","last_change":"2024-05-08T04:08:20.691470+0000","last_active":"2024-05-08T04:08:23.857745+0000","last_peered":"2024-05-08T04:08:23.857745+0000","last_clean":"2024-05-08T04:08:23.857745+0000","last_became_active":"2024-05-08T04:08:20.691204+0000","last_became_peered":"2024-05-08T04:08:20.691204+0000","last_unstale":"2024-05-08T04:08:23.857745+0000","last_undegraded":"2024-05-08T04:08:23.857745+0000","last_fullsized":"2024-05-08T04:08:23.857745+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:19.679313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:19.679313+0000","last_clean_scrub_stamp":"2024-05-08T04:08:19.679313+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:52:31.185659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":10222,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":577937,"data_compressed":5111,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":49009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542148,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":49009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":147456,"data_stored":44381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":155648,"data_stored":49009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":49009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":577937,"data_compressed":5111,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":44381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5111,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5111,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:08:26.030 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T04:08:26.030 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-08T04:08:26.030 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-08T04:08:26.031 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-08T04:08:26.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.127+0000 7f331166b640 1 Processor -- start 2024-05-08T04:08:26.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.127+0000 7f331166b640 1 -- start start 2024-05-08T04:08:26.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.127+0000 7f331166b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f330c0b62c0 0x7f330c0aab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:26.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.128+0000 7f331166b640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f330c140f30 0x7f330c147410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:26.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.128+0000 7f331166b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f330c0aa8c0 con 0x7f330c140ad0 2024-05-08T04:08:26.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.128+0000 7f331166b640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7f330c0b66a0 con 0x7f330c140f30 2024-05-08T04:08:26.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.128+0000 7f331166b640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f330c0ab090 con 0x7f330c0b62c0 2024-05-08T04:08:26.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.128+0000 7f330b7fe640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f330c140f30 0x7f330c147410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:26.132 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.128+0000 7f330affd640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f330c0b62c0 0x7f330c0aab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:26.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.128+0000 7f330affd640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f330c0b62c0 0x7f330c0aab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:57780/0 (socket says 172.21.15.170:57780) 2024-05-08T04:08:26.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.128+0000 7f330b7fe640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f330c140f30 0x7f330c147410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41092/0 (socket says 172.21.15.170:41092) 2024-05-08T04:08:26.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.128+0000 7f330affd640 1 -- 172.21.15.170:0/3993122376 learned_addr learned my addr 172.21.15.170:0/3993122376 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:26.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.128+0000 7f330affd640 1 -- 172.21.15.170:0/3993122376 >> v2:172.21.15.170:3300/0 conn(0x7f330c140f30 msgr2=0x7f330c147410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:26.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.128+0000 7f330affd640 1 --2- 172.21.15.170:0/3993122376 >> v2:172.21.15.170:3300/0 conn(0x7f330c140f30 0x7f330c147410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:26.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.128+0000 7f330affd640 1 -- 172.21.15.170:0/3993122376 >> v1:172.21.15.187:6789/0 conn(0x7f330c140ad0 legacy=0x7f330c146d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:26.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.128+0000 7f330affd640 1 -- 172.21.15.170:0/3993122376 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f330c0ac790 con 0x7f330c0b62c0 2024-05-08T04:08:26.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.128+0000 7f330b7fe640 1 --2- 172.21.15.170:0/3993122376 >> v2:172.21.15.170:3300/0 conn(0x7f330c140f30 0x7f330c147410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:08:26.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.129+0000 7f330affd640 1 --2- 172.21.15.170:0/3993122376 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f330c0b62c0 0x7f330c0aab50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f32fc009870 tx=0x7f32fc02b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67be47ccdbc01945 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:26.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.129+0000 7f3309ffb640 1 -- 172.21.15.170:0/3993122376 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc0317c0 con 0x7f330c0b62c0 2024-05-08T04:08:26.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.129+0000 7f3309ffb640 1 -- 172.21.15.170:0/3993122376 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32fc037070 con 0x7f330c0b62c0 2024-05-08T04:08:26.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.129+0000 7f3309ffb640 1 -- 172.21.15.170:0/3993122376 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32fc031b80 con 0x7f330c0b62c0 2024-05-08T04:08:26.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.129+0000 7f331166b640 1 -- 172.21.15.170:0/3993122376 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f330c0b62c0 msgr2=0x7f330c0aab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:26.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.129+0000 7f331166b640 1 --2- 172.21.15.170:0/3993122376 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f330c0b62c0 0x7f330c0aab50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f32fc009870 tx=0x7f32fc02b860 comp rx=0 tx=0).stop 2024-05-08T04:08:26.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.129+0000 7f331166b640 1 -- 172.21.15.170:0/3993122376 shutdown_connections 2024-05-08T04:08:26.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.129+0000 7f331166b640 1 --2- 172.21.15.170:0/3993122376 >> v2:172.21.15.170:3300/0 conn(0x7f330c140f30 0x7f330c147410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:26.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.129+0000 7f331166b640 1 --2- 172.21.15.170:0/3993122376 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f330c0b62c0 0x7f330c0aab50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:26.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.129+0000 7f331166b640 1 -- 172.21.15.170:0/3993122376 >> 172.21.15.170:0/3993122376 conn(0x7f330c076050 msgr2=0x7f330c076460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:26.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.130+0000 7f331166b640 1 -- 172.21.15.170:0/3993122376 shutdown_connections 2024-05-08T04:08:26.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.130+0000 7f331166b640 1 -- 172.21.15.170:0/3993122376 wait complete. 2024-05-08T04:08:26.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.131+0000 7f331166b640 1 Processor -- start 2024-05-08T04:08:26.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.131+0000 7f331166b640 1 -- start start 2024-05-08T04:08:26.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.131+0000 7f331166b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f330c0b62c0 0x7f330c1487f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:26.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.131+0000 7f331166b640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f330c140ad0 0x7f330c149360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:26.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.131+0000 7f331166b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f330c140f30 0x7f330c13a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:26.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.131+0000 7f331166b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f330c1413c0 con 0x7f330c140f30 2024-05-08T04:08:26.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.131+0000 7f331166b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f330c147950 con 0x7f330c0b62c0 2024-05-08T04:08:26.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.131+0000 7f331166b640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f330c14e090 con 0x7f330c140ad0 2024-05-08T04:08:26.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.131+0000 7f330a7fc640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f330c140ad0 0x7f330c149360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:26.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.131+0000 7f330b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f330c140f30 0x7f330c13a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:26.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.131+0000 7f330b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f330c140f30 0x7f330c13a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41094/0 (socket says 172.21.15.170:41094) 2024-05-08T04:08:26.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.131+0000 7f330a7fc640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f330c140ad0 0x7f330c149360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:57784/0 (socket says 172.21.15.170:57784) 2024-05-08T04:08:26.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.131+0000 7f330b7fe640 1 -- 172.21.15.170:0/4120438496 learned_addr learned my addr 172.21.15.170:0/4120438496 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:26.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.132+0000 7f330affd640 1 --2- 172.21.15.170:0/4120438496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f330c0b62c0 0x7f330c1487f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:26.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.132+0000 7f330b7fe640 1 -- 172.21.15.170:0/4120438496 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f330c140ad0 msgr2=0x7f330c149360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:26.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.132+0000 7f330b7fe640 1 --2- 172.21.15.170:0/4120438496 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f330c140ad0 0x7f330c149360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:26.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.132+0000 7f330b7fe640 1 -- 172.21.15.170:0/4120438496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f330c0b62c0 msgr2=0x7f330c1487f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:26.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.132+0000 7f330b7fe640 1 --2- 172.21.15.170:0/4120438496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f330c0b62c0 0x7f330c1487f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:26.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.132+0000 7f330b7fe640 1 -- 172.21.15.170:0/4120438496 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f330c14a900 con 0x7f330c140f30 2024-05-08T04:08:26.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.132+0000 7f330a7fc640 1 --2- 172.21.15.170:0/4120438496 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f330c140ad0 0x7f330c149360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:08:26.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.132+0000 7f330b7fe640 1 --2- 172.21.15.170:0/4120438496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f330c140f30 0x7f330c13a260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3304009e60 tx=0x7f330400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:26.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.132+0000 7f32f37fe640 1 -- 172.21.15.170:0/4120438496 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3304018030 con 0x7f330c140f30 2024-05-08T04:08:26.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.132+0000 7f32f37fe640 1 -- 172.21.15.170:0/4120438496 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f330400cdf0 con 0x7f330c140f30 2024-05-08T04:08:26.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.132+0000 7f331166b640 1 -- 172.21.15.170:0/4120438496 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f330c14abe0 con 0x7f330c140f30 2024-05-08T04:08:26.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.132+0000 7f32f37fe640 1 -- 172.21.15.170:0/4120438496 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3304003bb0 con 0x7f330c140f30 2024-05-08T04:08:26.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.132+0000 7f331166b640 1 -- 172.21.15.170:0/4120438496 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f330c13a7a0 con 0x7f330c140f30 2024-05-08T04:08:26.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.133+0000 7f331166b640 1 -- 172.21.15.170:0/4120438496 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f330c06b400 con 0x7f330c140f30 2024-05-08T04:08:26.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.133+0000 7f32f37fe640 1 -- 172.21.15.170:0/4120438496 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f330401b070 con 0x7f330c140f30 2024-05-08T04:08:26.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.134+0000 7f32f37fe640 1 --2- 172.21.15.170:0/4120438496 >> v2:172.21.15.187:6816/101007897 conn(0x7f32f8076290 0x7f32f8078750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:26.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.134+0000 7f32f37fe640 1 -- 172.21.15.170:0/4120438496 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4090+0+0 (secure 0 0 0) 0x7f33040978c0 con 0x7f330c140f30 2024-05-08T04:08:26.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.134+0000 7f330affd640 1 --2- 172.21.15.170:0/4120438496 >> v2:172.21.15.187:6816/101007897 conn(0x7f32f8076290 0x7f32f8078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:26.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.135+0000 7f330affd640 1 --2- 172.21.15.170:0/4120438496 >> v2:172.21.15.187:6816/101007897 conn(0x7f32f8076290 0x7f32f8078750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f32fc036000 tx=0x7f32fc03d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:26.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.138+0000 7f32f37fe640 1 -- 172.21.15.170:0/4120438496 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3304010040 con 0x7f330c140f30 2024-05-08T04:08:26.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.356+0000 7f331166b640 1 -- 172.21.15.170:0/4120438496 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f330c072bd0 con 0x7f330c140f30 2024-05-08T04:08:26.358 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.356+0000 7f32f37fe640 1 -- 172.21.15.170:0/4120438496 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f3304061870 con 0x7f330c140f30 2024-05-08T04:08:26.359 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:08:26.359 INFO:teuthology.orchestra.run.smithi170.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-08T04:08:26.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.358+0000 7f331166b640 1 -- 172.21.15.170:0/4120438496 >> v2:172.21.15.187:6816/101007897 conn(0x7f32f8076290 msgr2=0x7f32f8078750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:26.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.358+0000 7f331166b640 1 --2- 172.21.15.170:0/4120438496 >> v2:172.21.15.187:6816/101007897 conn(0x7f32f8076290 0x7f32f8078750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f32fc036000 tx=0x7f32fc03d040 comp rx=0 tx=0).stop 2024-05-08T04:08:26.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.358+0000 7f331166b640 1 -- 172.21.15.170:0/4120438496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f330c140f30 msgr2=0x7f330c13a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:26.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.358+0000 7f331166b640 1 --2- 172.21.15.170:0/4120438496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f330c140f30 0x7f330c13a260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3304009e60 tx=0x7f330400c490 comp rx=0 tx=0).stop 2024-05-08T04:08:26.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.358+0000 7f331166b640 1 -- 172.21.15.170:0/4120438496 shutdown_connections 2024-05-08T04:08:26.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.358+0000 7f331166b640 1 --2- 172.21.15.170:0/4120438496 >> v2:172.21.15.187:6816/101007897 conn(0x7f32f8076290 0x7f32f8078750 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:26.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.358+0000 7f331166b640 1 --2- 172.21.15.170:0/4120438496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f330c140f30 0x7f330c13a260 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:26.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.358+0000 7f331166b640 1 --2- 172.21.15.170:0/4120438496 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f330c140ad0 0x7f330c149360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:26.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.358+0000 7f331166b640 1 --2- 172.21.15.170:0/4120438496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f330c0b62c0 0x7f330c1487f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:26.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.358+0000 7f331166b640 1 -- 172.21.15.170:0/4120438496 >> 172.21.15.170:0/4120438496 conn(0x7f330c076050 msgr2=0x7f330c076430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:26.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.358+0000 7f331166b640 1 -- 172.21.15.170:0/4120438496 shutdown_connections 2024-05-08T04:08:26.361 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.358+0000 7f331166b640 1 -- 172.21.15.170:0/4120438496 wait complete. 2024-05-08T04:08:26.372 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-08T04:08:26.372 INFO:teuthology.run_tasks:Running task exec... 2024-05-08T04:08:26.382 INFO:teuthology.task.exec:Executing custom commands... 2024-05-08T04:08:26.382 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi170.front.sepia.ceph.com 2024-05-08T04:08:26.382 DEBUG:teuthology.orchestra.run.smithi170:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd pool create cephsqlite' 2024-05-08T04:08:26.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.479+0000 7fd1bd12d640 1 Processor -- start 2024-05-08T04:08:26.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.479+0000 7fd1bd12d640 1 -- start start 2024-05-08T04:08:26.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.479+0000 7fd1bd12d640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fd1b80b2290 0x7fd1b8146c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:26.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.479+0000 7fd1bd12d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd1b80aa4a0 0x7fd1b80b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:26.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.479+0000 7fd1bd12d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd1b80b5dc0 0x7fd1b80b6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:26.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.479+0000 7fd1bd12d640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7fd1b81471d0 con 0x7fd1b80b2290 2024-05-08T04:08:26.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.479+0000 7fd1bd12d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd1b80b8cd0 con 0x7fd1b80b5dc0 2024-05-08T04:08:26.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.479+0000 7fd1bd12d640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fd1b80ab390 con 0x7fd1b80aa4a0 2024-05-08T04:08:26.484 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.480+0000 7fd1b659c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd1b80aa4a0 0x7fd1b80b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:26.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.480+0000 7fd1b6d9d640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fd1b80b2290 0x7fd1b8146c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:26.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.480+0000 7fd1b659c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd1b80aa4a0 0x7fd1b80b2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:57790/0 (socket says 172.21.15.170:57790) 2024-05-08T04:08:26.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.480+0000 7fd1b6d9d640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fd1b80b2290 0x7fd1b8146c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41102/0 (socket says 172.21.15.170:41102) 2024-05-08T04:08:26.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.480+0000 7fd1b659c640 1 -- 172.21.15.170:0/2903084539 learned_addr learned my addr 172.21.15.170:0/2903084539 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:26.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.480+0000 7fd1b759e640 1 --2- 172.21.15.170:0/2903084539 >> v2:172.21.15.187:3300/0 conn(0x7fd1b80b5dc0 0x7fd1b80b6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:26.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.480+0000 7fd1b6d9d640 1 -- 172.21.15.170:0/2903084539 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd1b80aa4a0 msgr2=0x7fd1b80b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:26.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.480+0000 7fd1b6d9d640 1 --2- 172.21.15.170:0/2903084539 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd1b80aa4a0 0x7fd1b80b2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:26.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.480+0000 7fd1b6d9d640 1 -- 172.21.15.170:0/2903084539 >> v2:172.21.15.187:3300/0 conn(0x7fd1b80b5dc0 msgr2=0x7fd1b80b6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:26.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.480+0000 7fd1b6d9d640 1 --2- 172.21.15.170:0/2903084539 >> v2:172.21.15.187:3300/0 conn(0x7fd1b80b5dc0 0x7fd1b80b6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:26.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.480+0000 7fd1b6d9d640 1 -- 172.21.15.170:0/2903084539 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b80b33c0 con 0x7fd1b80b2290 2024-05-08T04:08:26.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.480+0000 7fd1b659c640 1 --2- 172.21.15.170:0/2903084539 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd1b80aa4a0 0x7fd1b80b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:08:26.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.480+0000 7fd1b759e640 1 --2- 172.21.15.170:0/2903084539 >> v2:172.21.15.187:3300/0 conn(0x7fd1b80b5dc0 0x7fd1b80b6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:08:26.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.481+0000 7fd1b6d9d640 1 --2- 172.21.15.170:0/2903084539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1b80b2290 0x7fd1b8146c90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd1b80b85e0 tx=0x7fd1a802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db0b75082e38ee0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:26.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.481+0000 7fd1b5d9b640 1 -- 172.21.15.170:0/2903084539 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8038070 con 0x7fd1b80b2290 2024-05-08T04:08:26.485 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.481+0000 7fd1b5d9b640 1 -- 172.21.15.170:0/2903084539 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1a8031700 con 0x7fd1b80b2290 2024-05-08T04:08:26.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.481+0000 7fd1b5d9b640 1 -- 172.21.15.170:0/2903084539 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8031ad0 con 0x7fd1b80b2290 2024-05-08T04:08:26.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.481+0000 7fd1bd12d640 1 -- 172.21.15.170:0/2903084539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1b80b2290 msgr2=0x7fd1b8146c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:26.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.481+0000 7fd1bd12d640 1 --2- 172.21.15.170:0/2903084539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1b80b2290 0x7fd1b8146c90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd1b80b85e0 tx=0x7fd1a802b7a0 comp rx=0 tx=0).stop 2024-05-08T04:08:26.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.481+0000 7fd1bd12d640 1 -- 172.21.15.170:0/2903084539 shutdown_connections 2024-05-08T04:08:26.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.481+0000 7fd1bd12d640 1 --2- 172.21.15.170:0/2903084539 >> v2:172.21.15.187:3300/0 conn(0x7fd1b80b5dc0 0x7fd1b80b6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:26.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.481+0000 7fd1bd12d640 1 --2- 172.21.15.170:0/2903084539 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd1b80aa4a0 0x7fd1b80b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:26.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.481+0000 7fd1bd12d640 1 --2- 172.21.15.170:0/2903084539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1b80b2290 0x7fd1b8146c90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:26.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.481+0000 7fd1bd12d640 1 -- 172.21.15.170:0/2903084539 >> 172.21.15.170:0/2903084539 conn(0x7fd1b802cbb0 msgr2=0x7fd1b802cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:26.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.481+0000 7fd1bd12d640 1 -- 172.21.15.170:0/2903084539 shutdown_connections 2024-05-08T04:08:26.486 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.481+0000 7fd1bd12d640 1 -- 172.21.15.170:0/2903084539 wait complete. 2024-05-08T04:08:26.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.483+0000 7fd1bd12d640 1 Processor -- start 2024-05-08T04:08:26.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.483+0000 7fd1bd12d640 1 -- start start 2024-05-08T04:08:26.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.483+0000 7fd1bd12d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1b80aa4a0 0x7fd1b81cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:26.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.483+0000 7fd1bd12d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1b80b2290 0x7fd1b8143ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:26.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.483+0000 7fd1bd12d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd1b80b5dc0 0x7fd1b81dc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:26.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.483+0000 7fd1bd12d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b80ab0c0 con 0x7fd1b80b2290 2024-05-08T04:08:26.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.483+0000 7fd1bd12d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd1b80aaf40 con 0x7fd1b80aa4a0 2024-05-08T04:08:26.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.483+0000 7fd1bd12d640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fd1b80b2670 con 0x7fd1b80b5dc0 2024-05-08T04:08:26.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.483+0000 7fd1b659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1b80b2290 0x7fd1b8143ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:26.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.483+0000 7fd1b759e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd1b80b5dc0 0x7fd1b81dc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:26.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.483+0000 7fd1b659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1b80b2290 0x7fd1b8143ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41116/0 (socket says 172.21.15.170:41116) 2024-05-08T04:08:26.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.483+0000 7fd1b659c640 1 -- 172.21.15.170:0/250041554 learned_addr learned my addr 172.21.15.170:0/250041554 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:26.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.483+0000 7fd1b6d9d640 1 --2- 172.21.15.170:0/250041554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1b80aa4a0 0x7fd1b81cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:26.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.484+0000 7fd1b759e640 1 -- 172.21.15.170:0/250041554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1b80aa4a0 msgr2=0x7fd1b81cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:26.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.484+0000 7fd1b759e640 1 --2- 172.21.15.170:0/250041554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1b80aa4a0 0x7fd1b81cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:26.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.484+0000 7fd1b759e640 1 -- 172.21.15.170:0/250041554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1b80b2290 msgr2=0x7fd1b8143ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:26.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.484+0000 7fd1b759e640 1 --2- 172.21.15.170:0/250041554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1b80b2290 0x7fd1b8143ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:26.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.484+0000 7fd1b759e640 1 -- 172.21.15.170:0/250041554 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1b81cdbd0 con 0x7fd1b80b5dc0 2024-05-08T04:08:26.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.484+0000 7fd1b659c640 1 --2- 172.21.15.170:0/250041554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1b80b2290 0x7fd1b8143ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:08:26.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.484+0000 7fd1b759e640 1 --2- 172.21.15.170:0/250041554 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd1b80b5dc0 0x7fd1b81dc110 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0007d70 tx=0x7fd1b000d4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:26.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.484+0000 7fd19f7fe640 1 -- 172.21.15.170:0/250041554 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b001a070 con 0x7fd1b80b5dc0 2024-05-08T04:08:26.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.484+0000 7fd19f7fe640 1 -- 172.21.15.170:0/250041554 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1b0003990 con 0x7fd1b80b5dc0 2024-05-08T04:08:26.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.484+0000 7fd1bd12d640 1 -- 172.21.15.170:0/250041554 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1b81cdeb0 con 0x7fd1b80b5dc0 2024-05-08T04:08:26.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.484+0000 7fd19f7fe640 1 -- 172.21.15.170:0/250041554 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0003d80 con 0x7fd1b80b5dc0 2024-05-08T04:08:26.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.484+0000 7fd1bd12d640 1 -- 172.21.15.170:0/250041554 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1b813ff20 con 0x7fd1b80b5dc0 2024-05-08T04:08:26.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.485+0000 7fd1bd12d640 1 -- 172.21.15.170:0/250041554 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1b81401e0 con 0x7fd1b80b5dc0 2024-05-08T04:08:26.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.485+0000 7fd19f7fe640 1 -- 172.21.15.170:0/250041554 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7fd1b001d070 con 0x7fd1b80b5dc0 2024-05-08T04:08:26.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.486+0000 7fd19f7fe640 1 --2- 172.21.15.170:0/250041554 >> v2:172.21.15.187:6816/101007897 conn(0x7fd1a4076290 0x7fd1a4078750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:26.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.486+0000 7fd19f7fe640 1 -- 172.21.15.170:0/250041554 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(18..18 src has 1..18) ==== 4090+0+0 (secure 0 0 0) 0x7fd1b00970b0 con 0x7fd1b80b5dc0 2024-05-08T04:08:26.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.486+0000 7fd1b6d9d640 1 --2- 172.21.15.170:0/250041554 >> v2:172.21.15.187:6816/101007897 conn(0x7fd1a4076290 0x7fd1a4078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:26.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.487+0000 7fd1b6d9d640 1 --2- 172.21.15.170:0/250041554 >> v2:172.21.15.187:6816/101007897 conn(0x7fd1a4076290 0x7fd1a4078750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8035000 tx=0x7fd1a8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:26.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.490+0000 7fd19f7fe640 1 -- 172.21.15.170:0/250041554 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd1b0010070 con 0x7fd1b80b5dc0 2024-05-08T04:08:26.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.671+0000 7fd1bd12d640 1 -- 172.21.15.170:0/250041554 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephsqlite"} v 0) -- 0x7fd1b80778a0 con 0x7fd1b80b5dc0 2024-05-08T04:08:26.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.716+0000 7fd19f7fe640 1 -- 172.21.15.170:0/250041554 <== mon.2 v2:172.21.15.170:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephsqlite"}]=0 pool 'cephsqlite' created v19) ==== 110+0+0 (secure 0 0 0) 0x7fd1b0060a10 con 0x7fd1b80b5dc0 2024-05-08T04:08:26.726 INFO:teuthology.orchestra.run.smithi170.stderr:pool 'cephsqlite' created 2024-05-08T04:08:26.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.720+0000 7fd1bd12d640 1 -- 172.21.15.170:0/250041554 >> v2:172.21.15.187:6816/101007897 conn(0x7fd1a4076290 msgr2=0x7fd1a4078750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:26.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.720+0000 7fd1bd12d640 1 --2- 172.21.15.170:0/250041554 >> v2:172.21.15.187:6816/101007897 conn(0x7fd1a4076290 0x7fd1a4078750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8035000 tx=0x7fd1a8008030 comp rx=0 tx=0).stop 2024-05-08T04:08:26.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.720+0000 7fd1bd12d640 1 -- 172.21.15.170:0/250041554 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd1b80b5dc0 msgr2=0x7fd1b81dc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:26.726 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.720+0000 7fd1bd12d640 1 --2- 172.21.15.170:0/250041554 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd1b80b5dc0 0x7fd1b81dc110 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0007d70 tx=0x7fd1b000d4a0 comp rx=0 tx=0).stop 2024-05-08T04:08:26.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.720+0000 7fd1bd12d640 1 -- 172.21.15.170:0/250041554 shutdown_connections 2024-05-08T04:08:26.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.720+0000 7fd1bd12d640 1 --2- 172.21.15.170:0/250041554 >> v2:172.21.15.187:6816/101007897 conn(0x7fd1a4076290 0x7fd1a4078750 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:26.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.720+0000 7fd1bd12d640 1 --2- 172.21.15.170:0/250041554 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd1b80b5dc0 0x7fd1b81dc110 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:26.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.720+0000 7fd1bd12d640 1 --2- 172.21.15.170:0/250041554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1b80b2290 0x7fd1b8143ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:26.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.720+0000 7fd1bd12d640 1 --2- 172.21.15.170:0/250041554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd1b80aa4a0 0x7fd1b81cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:26.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.720+0000 7fd1bd12d640 1 -- 172.21.15.170:0/250041554 >> 172.21.15.170:0/250041554 conn(0x7fd1b802cbb0 msgr2=0x7fd1b80b96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:26.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.721+0000 7fd1bd12d640 1 -- 172.21.15.170:0/250041554 shutdown_connections 2024-05-08T04:08:26.727 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.721+0000 7fd1bd12d640 1 -- 172.21.15.170:0/250041554 wait complete. 2024-05-08T04:08:26.737 DEBUG:teuthology.orchestra.run.smithi170:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph auth get-or-create client.libcephsqlite mon '"'"'profile simple-rados-client-with-blocklist'"'"' osd '"'"'allow rwx pool=cephsqlite'"'"' >> /etc/ceph/ceph.keyring' 2024-05-08T04:08:26.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.838+0000 7f44384c3640 1 Processor -- start 2024-05-08T04:08:26.840 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.838+0000 7f44384c3640 1 -- start start 2024-05-08T04:08:26.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.839+0000 7f44384c3640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f44300b63a0 0x7f44300aaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:26.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.839+0000 7f44384c3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4430140a50 0x7f4430140e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:26.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.839+0000 7f44384c3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4430144380 con 0x7f44301437e0 2024-05-08T04:08:26.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.839+0000 7f44384c3640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7f44300b6780 con 0x7f44300b63a0 2024-05-08T04:08:26.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.839+0000 7f44384c3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f44300ac900 con 0x7f4430140a50 2024-05-08T04:08:26.842 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.839+0000 7f443625f640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f44300b63a0 0x7f44300aaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:26.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.839+0000 7f443625f640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f44300b63a0 0x7f44300aaa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41120/0 (socket says 172.21.15.170:41120) 2024-05-08T04:08:26.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.839+0000 7f4436a60640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f44301437e0 0x7f4430143c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.170:43102/0 (socket says 172.21.15.170:43102) 2024-05-08T04:08:26.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.839+0000 7f443625f640 1 -- 172.21.15.170:0/3722743067 learned_addr learned my addr 172.21.15.170:0/3722743067 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:26.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.839+0000 7f4435a5e640 1 --2- 172.21.15.170:0/3722743067 >> v2:172.21.15.187:3300/0 conn(0x7f4430140a50 0x7f4430140e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:26.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.839+0000 7f443625f640 1 -- 172.21.15.170:0/3722743067 >> v2:172.21.15.187:3300/0 conn(0x7f4430140a50 msgr2=0x7f4430140e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:08:26.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.839+0000 7f443625f640 1 --2- 172.21.15.170:0/3722743067 >> v2:172.21.15.187:3300/0 conn(0x7f4430140a50 0x7f4430140e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:26.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.839+0000 7f443625f640 1 -- 172.21.15.170:0/3722743067 >> v1:172.21.15.187:6789/0 conn(0x7f44301437e0 legacy=0x7f4430143c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:26.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.839+0000 7f443625f640 1 -- 172.21.15.170:0/3722743067 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44301413d0 con 0x7f44300b63a0 2024-05-08T04:08:26.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.839+0000 7f443525d640 1 -- 172.21.15.170:0/3722743067 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1714803535 0 0) 0x7f4430144380 con 0x7f44301437e0 2024-05-08T04:08:26.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.840+0000 7f443625f640 1 --2- 172.21.15.170:0/3722743067 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44300b63a0 0x7f44300aaa00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f442c009870 tx=0x7f442c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c39751b8fa0e2c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:26.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.840+0000 7f443525d640 1 -- 172.21.15.170:0/3722743067 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c034070 con 0x7f44300b63a0 2024-05-08T04:08:26.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.840+0000 7f443525d640 1 -- 172.21.15.170:0/3722743067 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f442c030860 con 0x7f44300b63a0 2024-05-08T04:08:26.843 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.840+0000 7f443525d640 1 -- 172.21.15.170:0/3722743067 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c030c30 con 0x7f44300b63a0 2024-05-08T04:08:26.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.841+0000 7f44384c3640 1 -- 172.21.15.170:0/3722743067 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44300b63a0 msgr2=0x7f44300aaa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:26.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.841+0000 7f44384c3640 1 --2- 172.21.15.170:0/3722743067 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44300b63a0 0x7f44300aaa00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f442c009870 tx=0x7f442c02b9a0 comp rx=0 tx=0).stop 2024-05-08T04:08:26.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.841+0000 7f44384c3640 1 -- 172.21.15.170:0/3722743067 shutdown_connections 2024-05-08T04:08:26.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.841+0000 7f44384c3640 1 --2- 172.21.15.170:0/3722743067 >> v2:172.21.15.187:3300/0 conn(0x7f4430140a50 0x7f4430140e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:26.846 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.841+0000 7f44384c3640 1 --2- 172.21.15.170:0/3722743067 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44300b63a0 0x7f44300aaa00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:26.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.841+0000 7f44384c3640 1 -- 172.21.15.170:0/3722743067 >> 172.21.15.170:0/3722743067 conn(0x7f443002ce20 msgr2=0x7f443002d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:26.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.841+0000 7f44384c3640 1 -- 172.21.15.170:0/3722743067 shutdown_connections 2024-05-08T04:08:26.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.841+0000 7f44384c3640 1 -- 172.21.15.170:0/3722743067 wait complete. 2024-05-08T04:08:26.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.842+0000 7f44384c3640 1 Processor -- start 2024-05-08T04:08:26.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.842+0000 7f44384c3640 1 -- start start 2024-05-08T04:08:26.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.842+0000 7f44384c3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44300b63a0 0x7f443013ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:26.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.842+0000 7f44384c3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4430140a50 0x7f44300b2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:26.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.842+0000 7f44384c3640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f44301437e0 0x7f44300b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:26.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.842+0000 7f44384c3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f44300aaf40 con 0x7f44300b63a0 2024-05-08T04:08:26.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.842+0000 7f44384c3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f44300ab3a0 con 0x7f4430140a50 2024-05-08T04:08:26.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.842+0000 7f44384c3640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f44300ab0c0 con 0x7f44301437e0 2024-05-08T04:08:26.847 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.842+0000 7f443625f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44300b63a0 0x7f443013ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:26.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.842+0000 7f443625f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44300b63a0 0x7f443013ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41126/0 (socket says 172.21.15.170:41126) 2024-05-08T04:08:26.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.842+0000 7f443625f640 1 -- 172.21.15.170:0/1122794008 learned_addr learned my addr 172.21.15.170:0/1122794008 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:08:26.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.842+0000 7f4436a60640 1 --2- 172.21.15.170:0/1122794008 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f44301437e0 0x7f44300b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:26.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.843+0000 7f4435a5e640 1 --2- 172.21.15.170:0/1122794008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4430140a50 0x7f44300b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:26.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.843+0000 7f443625f640 1 -- 172.21.15.170:0/1122794008 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f44301437e0 msgr2=0x7f44300b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:26.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.843+0000 7f443625f640 1 --2- 172.21.15.170:0/1122794008 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f44301437e0 0x7f44300b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:26.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.843+0000 7f443625f640 1 -- 172.21.15.170:0/1122794008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4430140a50 msgr2=0x7f44300b2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:26.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.843+0000 7f443625f640 1 --2- 172.21.15.170:0/1122794008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4430140a50 0x7f44300b2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:26.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.843+0000 7f443625f640 1 -- 172.21.15.170:0/1122794008 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44300b2ee0 con 0x7f44300b63a0 2024-05-08T04:08:26.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.843+0000 7f443625f640 1 --2- 172.21.15.170:0/1122794008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44300b63a0 0x7f443013ee10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f442c000c00 tx=0x7f442c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:26.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.843+0000 7f4426ffd640 1 -- 172.21.15.170:0/1122794008 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c034040 con 0x7f44300b63a0 2024-05-08T04:08:26.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.843+0000 7f4426ffd640 1 -- 172.21.15.170:0/1122794008 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f442c030660 con 0x7f44300b63a0 2024-05-08T04:08:26.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.843+0000 7f44384c3640 1 -- 172.21.15.170:0/1122794008 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f443013c610 con 0x7f44300b63a0 2024-05-08T04:08:26.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.843+0000 7f4426ffd640 1 -- 172.21.15.170:0/1122794008 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c039d40 con 0x7f44300b63a0 2024-05-08T04:08:26.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.843+0000 7f44384c3640 1 -- 172.21.15.170:0/1122794008 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f443013c150 con 0x7f44300b63a0 2024-05-08T04:08:26.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.844+0000 7f44384c3640 1 -- 172.21.15.170:0/1122794008 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44300778a0 con 0x7f44300b63a0 2024-05-08T04:08:26.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.845+0000 7f4426ffd640 1 -- 172.21.15.170:0/1122794008 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f442c04b060 con 0x7f44300b63a0 2024-05-08T04:08:26.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.845+0000 7f4426ffd640 1 --2- 172.21.15.170:0/1122794008 >> v2:172.21.15.187:6816/101007897 conn(0x7f4414076290 0x7f4414078750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:08:26.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.845+0000 7f4426ffd640 1 -- 172.21.15.170:0/1122794008 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4439+0+0 (secure 0 0 0) 0x7f442c0b8c70 con 0x7f44300b63a0 2024-05-08T04:08:26.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.845+0000 7f4435a5e640 1 --2- 172.21.15.170:0/1122794008 >> v2:172.21.15.187:6816/101007897 conn(0x7f4414076290 0x7f4414078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:08:26.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.846+0000 7f4435a5e640 1 --2- 172.21.15.170:0/1122794008 >> v2:172.21.15.187:6816/101007897 conn(0x7f4414076290 0x7f4414078750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f44301437b0 tx=0x7f4420009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:08:26.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:26.850+0000 7f4426ffd640 1 -- 172.21.15.170:0/1122794008 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f442c082500 con 0x7f44300b63a0 2024-05-08T04:08:27.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:27.099+0000 7f44384c3640 1 -- 172.21.15.170:0/1122794008 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.libcephsqlite", "caps": ["mon", "profile simple-rados-client-with-blocklist", "osd", "allow rwx pool=cephsqlite"]} v 0) -- 0x7f443013c3e0 con 0x7f44300b63a0 2024-05-08T04:08:27.103 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:27.101+0000 7f4426ffd640 1 -- 172.21.15.170:0/1122794008 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.libcephsqlite", "caps": ["mon", "profile simple-rados-client-with-blocklist", "osd", "allow rwx pool=cephsqlite"]}]=0 v3) ==== 199+0+71 (secure 0 0 0) 0x7f442c086800 con 0x7f44300b63a0 2024-05-08T04:08:27.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:27.104+0000 7f44384c3640 1 -- 172.21.15.170:0/1122794008 >> v2:172.21.15.187:6816/101007897 conn(0x7f4414076290 msgr2=0x7f4414078750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:27.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:27.104+0000 7f44384c3640 1 --2- 172.21.15.170:0/1122794008 >> v2:172.21.15.187:6816/101007897 conn(0x7f4414076290 0x7f4414078750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f44301437b0 tx=0x7f4420009190 comp rx=0 tx=0).stop 2024-05-08T04:08:27.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:27.104+0000 7f44384c3640 1 -- 172.21.15.170:0/1122794008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44300b63a0 msgr2=0x7f443013ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:08:27.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:27.104+0000 7f44384c3640 1 --2- 172.21.15.170:0/1122794008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44300b63a0 0x7f443013ee10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f442c000c00 tx=0x7f442c0046c0 comp rx=0 tx=0).stop 2024-05-08T04:08:27.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:27.104+0000 7f44384c3640 1 -- 172.21.15.170:0/1122794008 shutdown_connections 2024-05-08T04:08:27.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:27.104+0000 7f44384c3640 1 --2- 172.21.15.170:0/1122794008 >> v2:172.21.15.187:6816/101007897 conn(0x7f4414076290 0x7f4414078750 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:27.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:27.104+0000 7f44384c3640 1 --2- 172.21.15.170:0/1122794008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4430140a50 0x7f44300b2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:27.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:27.104+0000 7f44384c3640 1 --2- 172.21.15.170:0/1122794008 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f44301437e0 0x7f44300b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:27.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:27.104+0000 7f44384c3640 1 --2- 172.21.15.170:0/1122794008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44300b63a0 0x7f443013ee10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:08:27.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:27.105+0000 7f44384c3640 1 -- 172.21.15.170:0/1122794008 >> 172.21.15.170:0/1122794008 conn(0x7f443002ce20 msgr2=0x7f443013fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:08:27.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:27.105+0000 7f44384c3640 1 -- 172.21.15.170:0/1122794008 shutdown_connections 2024-05-08T04:08:27.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:08:27.105+0000 7f44384c3640 1 -- 172.21.15.170:0/1122794008 wait complete. 2024-05-08T04:08:27.118 INFO:teuthology.run_tasks:Running task exec... 2024-05-08T04:08:27.128 INFO:teuthology.task.exec:Executing custom commands... 2024-05-08T04:08:27.128 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi170.front.sepia.ceph.com 2024-05-08T04:08:27.128 DEBUG:teuthology.orchestra.run.smithi170:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph_test_libcephsqlite --id libcephsqlite --no-log-to-stderr' 2024-05-08T04:08:27.173 INFO:teuthology.orchestra.run.smithi170.stdout:[==========] Running 20 tests from 1 test suite. 2024-05-08T04:08:27.173 INFO:teuthology.orchestra.run.smithi170.stdout:[----------] Global test environment set-up. 2024-05-08T04:08:27.173 INFO:teuthology.orchestra.run.smithi170.stdout:[----------] 20 tests from CephSQLiteTest 2024-05-08T04:08:32.181 INFO:teuthology.orchestra.run.smithi170.stdout:[ RUN ] CephSQLiteTest.Create 2024-05-08T04:08:32.192 INFO:teuthology.orchestra.run.smithi170.stdout:using database: file:cephsqlite:/6d430eb7-7aab-406e-90d1-f92b74390de4.db?vfs=ceph 2024-05-08T04:08:32.192 INFO:teuthology.orchestra.run.smithi170.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-05-08T04:08:32.198 INFO:teuthology.orchestra.run.smithi170.stdout:CREATE TABLE foo (a INT); 2024-05-08T04:08:32.224 INFO:teuthology.orchestra.run.smithi170.stdout:[ OK ] CephSQLiteTest.Create (43 ms) 2024-05-08T04:08:32.225 INFO:teuthology.orchestra.run.smithi170.stdout:[ RUN ] CephSQLiteTest.InsertBulk4096 2024-05-08T04:08:32.234 INFO:teuthology.orchestra.run.smithi170.stdout:using database: file:cephsqlite:/49668f59-9223-4cf7-b115-b427835ef28e.db?vfs=ceph 2024-05-08T04:08:32.234 INFO:teuthology.orchestra.run.smithi170.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-05-08T04:08:32.239 INFO:teuthology.orchestra.run.smithi170.stdout:PRAGMA page_size = 4096;CREATE TABLE foo (a INT);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT RANDOM() FROM c LIMIT 1000000;PRAGMA page_size; 2024-05-08T04:08:34.681 INFO:teuthology.orchestra.run.smithi170.stdout:[ OK ] CephSQLiteTest.InsertBulk4096 (2457 ms) 2024-05-08T04:08:34.681 INFO:teuthology.orchestra.run.smithi170.stdout:[ RUN ] CephSQLiteTest.InsertBulk 2024-05-08T04:08:34.689 INFO:teuthology.orchestra.run.smithi170.stdout:using database: file:cephsqlite:/261793c4-01f9-43b6-b7ab-40f972ae795d.db?vfs=ceph 2024-05-08T04:08:34.689 INFO:teuthology.orchestra.run.smithi170.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-05-08T04:08:34.693 INFO:teuthology.orchestra.run.smithi170.stdout:CREATE TABLE foo (a INT);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT RANDOM() FROM c LIMIT 1000000; 2024-05-08T04:08:35.145 INFO:teuthology.orchestra.run.smithi170.stdout:[ OK ] CephSQLiteTest.InsertBulk (464 ms) 2024-05-08T04:08:35.145 INFO:teuthology.orchestra.run.smithi170.stdout:[ RUN ] CephSQLiteTest.UpdateBulk 2024-05-08T04:08:35.152 INFO:teuthology.orchestra.run.smithi170.stdout:using database: file:cephsqlite:/94a85bab-8f4a-467b-a45b-60c105ba5b0f.db?vfs=ceph 2024-05-08T04:08:35.152 INFO:teuthology.orchestra.run.smithi170.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-05-08T04:08:35.156 INFO:teuthology.orchestra.run.smithi170.stdout:CREATE TABLE foo (a INT);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT x FROM c LIMIT 1000000;SELECT SUM(a) FROM foo;UPDATE foo SET a = a+a;SELECT SUM(a) FROM foo; 2024-05-08T04:08:35.834 INFO:teuthology.orchestra.run.smithi170.stdout:[ OK ] CephSQLiteTest.UpdateBulk (689 ms) 2024-05-08T04:08:35.834 INFO:teuthology.orchestra.run.smithi170.stdout:[ RUN ] CephSQLiteTest.InsertRate 2024-05-08T04:08:35.841 INFO:teuthology.orchestra.run.smithi170.stdout:using database: file:cephsqlite:/04e70832-2c37-4b59-8b0f-48918db73a2a.db?vfs=ceph 2024-05-08T04:08:35.841 INFO:teuthology.orchestra.run.smithi170.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-05-08T04:08:35.844 INFO:teuthology.orchestra.run.smithi170.stdout:CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM()); 2024-05-08T04:08:37.099 INFO:teuthology.orchestra.run.smithi170.stdout:transactions per second: 80.7121 2024-05-08T04:08:37.099 INFO:teuthology.orchestra.run.smithi170.stdout:[ OK ] CephSQLiteTest.InsertRate (1265 ms) 2024-05-08T04:08:37.099 INFO:teuthology.orchestra.run.smithi170.stdout:[ RUN ] CephSQLiteTest.DatabaseShrink 2024-05-08T04:08:37.107 INFO:teuthology.orchestra.run.smithi170.stdout:using database: file:cephsqlite:/113626a6-429a-44b6-9b1f-5d615f7eb443.db?vfs=ceph 2024-05-08T04:08:37.107 INFO:teuthology.orchestra.run.smithi170.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-05-08T04:08:37.110 INFO:teuthology.orchestra.run.smithi170.stdout:CREATE TABLE foo (a INT);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT x FROM c LIMIT 1000000;DELETE FROM foo WHERE RANDOM()%4 < 3;VACUUM; 2024-05-08T04:08:38.395 INFO:teuthology.orchestra.run.smithi170.stdout:[ OK ] CephSQLiteTest.DatabaseShrink (1296 ms) 2024-05-08T04:08:38.396 INFO:teuthology.orchestra.run.smithi170.stdout:[ RUN ] CephSQLiteTest.InsertExclusiveRate 2024-05-08T04:08:38.403 INFO:teuthology.orchestra.run.smithi170.stdout:using database: file:cephsqlite:/7eb33f16-73ad-4340-92b9-e42381472b78.db?vfs=ceph 2024-05-08T04:08:38.403 INFO:teuthology.orchestra.run.smithi170.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-05-08T04:08:38.407 INFO:teuthology.orchestra.run.smithi170.stdout:PRAGMA locking_mode=EXCLUSIVE;CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM()); 2024-05-08T04:08:39.065 INFO:teuthology.orchestra.run.smithi170.stdout:transactions per second: 154.563 2024-05-08T04:08:39.067 INFO:teuthology.orchestra.run.smithi170.stdout:[ OK ] CephSQLiteTest.InsertExclusiveRate (672 ms) 2024-05-08T04:08:39.067 INFO:teuthology.orchestra.run.smithi170.stdout:[ RUN ] CephSQLiteTest.InsertExclusiveWALRate 2024-05-08T04:08:39.075 INFO:teuthology.orchestra.run.smithi170.stdout:using database: file:cephsqlite:/49584242-e55c-4ac0-8cad-1070342bde3c.db?vfs=ceph 2024-05-08T04:08:39.075 INFO:teuthology.orchestra.run.smithi170.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-05-08T04:08:39.078 INFO:teuthology.orchestra.run.smithi170.stdout:PRAGMA locking_mode=EXCLUSIVE;PRAGMA journal_mode=WAL;CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM()); 2024-05-08T04:08:39.452 INFO:teuthology.orchestra.run.smithi170.stdout:transactions per second: 289.862 2024-05-08T04:08:39.466 INFO:teuthology.orchestra.run.smithi170.stdout:[ OK ] CephSQLiteTest.InsertExclusiveWALRate (399 ms) 2024-05-08T04:08:39.466 INFO:teuthology.orchestra.run.smithi170.stdout:[ RUN ] CephSQLiteTest.WALTransactionSync 2024-05-08T04:08:39.473 INFO:teuthology.orchestra.run.smithi170.stdout:using database: file:cephsqlite:/615c7e11-ab58-4046-a821-6aeaeaf0f60f.db?vfs=ceph 2024-05-08T04:08:39.473 INFO:teuthology.orchestra.run.smithi170.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-05-08T04:08:39.476 INFO:teuthology.orchestra.run.smithi170.stdout:PRAGMA locking_mode=EXCLUSIVE;PRAGMA journal_mode=WAL;CREATE TABLE foo (a INT);INSERT INTO perf (v) VALUES (ceph_perf());BEGIN TRANSACTION;INSERT INTO foo (a) VALUES (RANDOM());END TRANSACTION;INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_vfs.opf_sync.avgcount' AND b.fullkey = '$.libcephsqlite_vfs.opf_sync.avgcount'; 2024-05-08T04:08:39.519 INFO:teuthology.orchestra.run.smithi170.stdout:[ OK ] CephSQLiteTest.WALTransactionSync (53 ms) 2024-05-08T04:08:39.519 INFO:teuthology.orchestra.run.smithi170.stdout:[ RUN ] CephSQLiteTest.PersistTransactionSync 2024-05-08T04:08:39.527 INFO:teuthology.orchestra.run.smithi170.stdout:using database: file:cephsqlite:/328d32df-4d58-4061-a152-13539b7d683d.db?vfs=ceph 2024-05-08T04:08:39.527 INFO:teuthology.orchestra.run.smithi170.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-05-08T04:08:39.530 INFO:teuthology.orchestra.run.smithi170.stdout:BEGIN TRANSACTION;CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM());END TRANSACTION;INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_vfs.opf_sync.avgcount' AND b.fullkey = '$.libcephsqlite_vfs.opf_sync.avgcount'; 2024-05-08T04:08:39.548 INFO:teuthology.orchestra.run.smithi170.stdout:[ OK ] CephSQLiteTest.PersistTransactionSync (29 ms) 2024-05-08T04:08:39.548 INFO:teuthology.orchestra.run.smithi170.stdout:[ RUN ] CephSQLiteTest.InsertExclusiveLock 2024-05-08T04:08:39.556 INFO:teuthology.orchestra.run.smithi170.stdout:using database: file:cephsqlite:/6aaf7dd5-005e-43a9-ac23-3da4efd9c256.db?vfs=ceph 2024-05-08T04:08:39.556 INFO:teuthology.orchestra.run.smithi170.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-05-08T04:08:39.559 INFO:teuthology.orchestra.run.smithi170.stdout:PRAGMA locking_mode=EXCLUSIVE;CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM());INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom, b.atom, a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_vfs.opf_lock.avgcount' AND b.fullkey = '$.libcephsqlite_vfs.opf_lock.avgcount';SELECT a.atom, b.atom, a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_striper.lock' AND b.fullkey = '$.libcephsqlite_striper.lock'; 2024-05-08T04:08:39.578 INFO:teuthology.orchestra.run.smithi170.stdout:[ OK ] CephSQLiteTest.InsertExclusiveLock (30 ms) 2024-05-08T04:08:39.578 INFO:teuthology.orchestra.run.smithi170.stdout:[ RUN ] CephSQLiteTest.TransactionSizeUpdate 2024-05-08T04:08:39.585 INFO:teuthology.orchestra.run.smithi170.stdout:using database: file:cephsqlite:/77b35401-5b4d-4b38-8eb6-4f2c4e1e3983.db?vfs=ceph 2024-05-08T04:08:39.586 INFO:teuthology.orchestra.run.smithi170.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-05-08T04:08:39.589 INFO:teuthology.orchestra.run.smithi170.stdout:BEGIN TRANSACTION;CREATE TABLE foo (a INT);INSERT INTO foo (a) VALUES (RANDOM());END TRANSACTION;INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom, b.atom, a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_striper.update_size' AND b.fullkey = '$.libcephsqlite_striper.update_size'; 2024-05-08T04:08:39.604 INFO:teuthology.orchestra.run.smithi170.stdout:[ OK ] CephSQLiteTest.TransactionSizeUpdate (26 ms) 2024-05-08T04:08:39.605 INFO:teuthology.orchestra.run.smithi170.stdout:[ RUN ] CephSQLiteTest.AllocatedGrowth 2024-05-08T04:08:39.612 INFO:teuthology.orchestra.run.smithi170.stdout:using database: file:cephsqlite:/a9921b76-b3f1-43b2-b0a8-502dfb42aff3.db?vfs=ceph 2024-05-08T04:08:39.612 INFO:teuthology.orchestra.run.smithi170.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-05-08T04:08:39.615 INFO:teuthology.orchestra.run.smithi170.stdout:CREATE TABLE foo (a BLOB);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT RANDOMBLOB(1<<20) FROM c LIMIT 1024;INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom, b.atom, a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_striper.update_allocated' AND b.fullkey = '$.libcephsqlite_striper.update_allocated'; 2024-05-08T04:08:46.664 INFO:teuthology.orchestra.run.smithi170.stdout:[ OK ] CephSQLiteTest.AllocatedGrowth (7060 ms) 2024-05-08T04:08:46.664 INFO:teuthology.orchestra.run.smithi170.stdout:[ RUN ] CephSQLiteTest.DeleteBulk 2024-05-08T04:08:46.672 INFO:teuthology.orchestra.run.smithi170.stdout:using database: file:cephsqlite:/c1268857-54fa-4c06-bd66-3c3c5e59b5c1.db?vfs=ceph 2024-05-08T04:08:46.672 INFO:teuthology.orchestra.run.smithi170.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-05-08T04:08:46.677 INFO:teuthology.orchestra.run.smithi170.stdout:CREATE TABLE foo (a INT);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT x FROM c LIMIT 1000000;DELETE FROM foo WHERE RANDOM()%2 == 0; 2024-05-08T04:08:47.382 INFO:teuthology.orchestra.run.smithi170.stdout:[ OK ] CephSQLiteTest.DeleteBulk (718 ms) 2024-05-08T04:08:47.382 INFO:teuthology.orchestra.run.smithi170.stdout:[ RUN ] CephSQLiteTest.DropMassive 2024-05-08T04:08:47.389 INFO:teuthology.orchestra.run.smithi170.stdout:using database: file:cephsqlite:/08a0a92d-42e9-4b8d-ae79-8ec33f5fe0fe.db?vfs=ceph 2024-05-08T04:08:47.390 INFO:teuthology.orchestra.run.smithi170.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-05-08T04:08:47.393 INFO:teuthology.orchestra.run.smithi170.stdout:CREATE TABLE foo (a BLOB);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO foo (a) SELECT RANDOMBLOB(1<<20) FROM c LIMIT 1024;DROP TABLE foo;VACUUM;INSERT INTO perf (v) VALUES (ceph_perf());SELECT a.atom, b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_striper.shrink' AND b.fullkey = '$.libcephsqlite_striper.shrink';SELECT a.atom-b.atom FROM p AS a, p AS b WHERE a.i = ? AND b.i = ? AND a.fullkey = '$.libcephsqlite_striper.shrink_bytes' AND b.fullkey = '$.libcephsqlite_striper.shrink_bytes'; 2024-05-08T04:09:16.424 INFO:teuthology.orchestra.run.smithi170.stdout:[ OK ] CephSQLiteTest.DropMassive (29041 ms) 2024-05-08T04:09:16.424 INFO:teuthology.orchestra.run.smithi170.stdout:[ RUN ] CephSQLiteTest.InsertMassiveVerify 2024-05-08T04:09:16.432 INFO:teuthology.orchestra.run.smithi170.stdout:using database: file:cephsqlite:/9c048667-d5f8-4ef4-9313-b4c9dce29a9f.db?vfs=ceph 2024-05-08T04:09:16.432 INFO:teuthology.orchestra.run.smithi170.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-05-08T04:09:16.437 INFO:teuthology.orchestra.run.smithi170.stdout:CREATE TABLE foo (a BLOB);CREATE TEMPORARY TABLE bar (a BLOB);WITH RECURSIVE c(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM c )INSERT INTO bar (a) SELECT RANDOMBLOB(1<<20) FROM c LIMIT 1024;SELECT a FROM bar;INSERT INTO foo (a) SELECT a FROM bar;SELECT a FROM foo; 2024-05-08T04:09:29.331 INFO:teuthology.orchestra.run.smithi170.stdout:[ OK ] CephSQLiteTest.InsertMassiveVerify (12908 ms) 2024-05-08T04:09:29.331 INFO:teuthology.orchestra.run.smithi170.stdout:[ RUN ] CephSQLiteTest.PerfValid 2024-05-08T04:09:29.340 INFO:teuthology.orchestra.run.smithi170.stdout:using database: file:cephsqlite:/221f4651-96e4-42d5-89c8-5e312931317b.db?vfs=ceph 2024-05-08T04:09:29.340 INFO:teuthology.orchestra.run.smithi170.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-05-08T04:09:29.345 INFO:teuthology.orchestra.run.smithi170.stdout:SELECT json_valid(ceph_perf()); 2024-05-08T04:09:29.346 INFO:teuthology.orchestra.run.smithi170.stdout:[ OK ] CephSQLiteTest.PerfValid (15 ms) 2024-05-08T04:09:29.346 INFO:teuthology.orchestra.run.smithi170.stdout:[ RUN ] CephSQLiteTest.StatusValid 2024-05-08T04:09:29.355 INFO:teuthology.orchestra.run.smithi170.stdout:using database: file:cephsqlite:/9063f1b4-5f37-485c-8ac9-bb4e42145a3f.db?vfs=ceph 2024-05-08T04:09:29.356 INFO:teuthology.orchestra.run.smithi170.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-05-08T04:09:29.361 INFO:teuthology.orchestra.run.smithi170.stdout:SELECT json_valid(ceph_status()); 2024-05-08T04:09:29.362 INFO:teuthology.orchestra.run.smithi170.stdout:[ OK ] CephSQLiteTest.StatusValid (16 ms) 2024-05-08T04:09:29.362 INFO:teuthology.orchestra.run.smithi170.stdout:[ RUN ] CephSQLiteTest.CurrentTime 2024-05-08T04:09:29.371 INFO:teuthology.orchestra.run.smithi170.stdout:using database: file:cephsqlite:/fff77a6b-e7e9-42f9-81b9-855d8508caeb.db?vfs=ceph 2024-05-08T04:09:29.371 INFO:teuthology.orchestra.run.smithi170.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-05-08T04:09:29.378 INFO:teuthology.orchestra.run.smithi170.stdout:SELECT strftime('%s', 'now'); 2024-05-08T04:09:29.379 INFO:teuthology.orchestra.run.smithi170.stdout:[ OK ] CephSQLiteTest.CurrentTime (17 ms) 2024-05-08T04:09:29.379 INFO:teuthology.orchestra.run.smithi170.stdout:[ RUN ] CephSQLiteTest.StatusFields 2024-05-08T04:09:29.387 INFO:teuthology.orchestra.run.smithi170.stdout:using database: file:cephsqlite:/33f3311a-1776-417f-9f2b-d26fedacfdde.db?vfs=ceph 2024-05-08T04:09:29.388 INFO:teuthology.orchestra.run.smithi170.stdout:PRAGMA journal_mode = PERSIST;PRAGMA page_size = 65536;PRAGMA cache_size = 32768;PRAGMA temp_store = memory;CREATE TEMPORARY TABLE perf (i INTEGER PRIMARY KEY, v TEXT);CREATE TEMPORARY VIEW p AS SELECT perf.i, J.* FROM perf, json_tree(perf.v) AS J;INSERT INTO perf (v) VALUES (ceph_perf()); 2024-05-08T04:09:29.393 INFO:teuthology.orchestra.run.smithi170.stdout:SELECT json_extract(ceph_status(), '$.addr');SELECT json_extract(ceph_status(), '$.id'); 2024-05-08T04:09:29.393 INFO:teuthology.orchestra.run.smithi170.stdout:172.21.15.170:0/3355036497 2024-05-08T04:09:29.393 INFO:teuthology.orchestra.run.smithi170.stdout:4452 2024-05-08T04:09:29.394 INFO:teuthology.orchestra.run.smithi170.stdout:[ OK ] CephSQLiteTest.StatusFields (15 ms) 2024-05-08T04:09:29.394 INFO:teuthology.orchestra.run.smithi170.stdout:[----------] 20 tests from CephSQLiteTest (57213 ms total) 2024-05-08T04:09:29.394 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:09:29.394 INFO:teuthology.orchestra.run.smithi170.stdout:[----------] Global test environment tear-down 2024-05-08T04:09:29.394 INFO:teuthology.orchestra.run.smithi170.stdout:[==========] 20 tests from 1 test suite ran. (62221 ms total) 2024-05-08T04:09:29.394 INFO:teuthology.orchestra.run.smithi170.stdout:[ PASSED ] 20 tests. 2024-05-08T04:09:29.438 INFO:teuthology.run_tasks:Running task workunit... 2024-05-08T04:09:29.453 INFO:tasks.workunit:Pulling workunits from ref 5ccf1c30cf724d5371f55439b3b3a90cb994fae2 2024-05-08T04:09:29.453 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-08T04:09:29.453 DEBUG:teuthology.orchestra.run.smithi170:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-08T04:09:29.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T04:09:29.469 INFO:teuthology.orchestra.run.smithi170.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-05-08T04:09:29.469 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-05-08T04:09:29.525 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-05-08T04:09:29.525 DEBUG:teuthology.orchestra.run.smithi170:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-05-08T04:09:29.580 INFO:tasks.workunit:timeout=3h 2024-05-08T04:09:29.580 INFO:tasks.workunit:cleanup=True 2024-05-08T04:09:29.581 DEBUG:teuthology.orchestra.run.smithi170:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 5ccf1c30cf724d5371f55439b3b3a90cb994fae2 2024-05-08T04:09:29.637 INFO:tasks.workunit.client.0.smithi170.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-08T04:11:18.295 INFO:tasks.workunit.client.0.smithi170.stderr:Note: switching to '5ccf1c30cf724d5371f55439b3b3a90cb994fae2'. 2024-05-08T04:11:18.296 INFO:tasks.workunit.client.0.smithi170.stderr: 2024-05-08T04:11:18.296 INFO:tasks.workunit.client.0.smithi170.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-08T04:11:18.296 INFO:tasks.workunit.client.0.smithi170.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-08T04:11:18.296 INFO:tasks.workunit.client.0.smithi170.stderr:state without impacting any branches by switching back to a branch. 2024-05-08T04:11:18.296 INFO:tasks.workunit.client.0.smithi170.stderr: 2024-05-08T04:11:18.296 INFO:tasks.workunit.client.0.smithi170.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-08T04:11:18.296 INFO:tasks.workunit.client.0.smithi170.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-08T04:11:18.296 INFO:tasks.workunit.client.0.smithi170.stderr: 2024-05-08T04:11:18.296 INFO:tasks.workunit.client.0.smithi170.stderr: git switch -c 2024-05-08T04:11:18.296 INFO:tasks.workunit.client.0.smithi170.stderr: 2024-05-08T04:11:18.297 INFO:tasks.workunit.client.0.smithi170.stderr:Or undo this operation with: 2024-05-08T04:11:18.297 INFO:tasks.workunit.client.0.smithi170.stderr: 2024-05-08T04:11:18.297 INFO:tasks.workunit.client.0.smithi170.stderr: git switch - 2024-05-08T04:11:18.297 INFO:tasks.workunit.client.0.smithi170.stderr: 2024-05-08T04:11:18.297 INFO:tasks.workunit.client.0.smithi170.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-08T04:11:18.297 INFO:tasks.workunit.client.0.smithi170.stderr: 2024-05-08T04:11:18.297 INFO:tasks.workunit.client.0.smithi170.stderr:HEAD is now at 5ccf1c30cf7 systemd: do not pass --cluster option 2024-05-08T04:11:18.298 DEBUG:teuthology.orchestra.run.smithi170:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-08T04:11:18.376 INFO:tasks.workunit.client.0.smithi170.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-08T04:11:18.379 INFO:tasks.workunit.client.0.smithi170.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-08T04:11:18.379 INFO:tasks.workunit.client.0.smithi170.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-08T04:11:18.957 INFO:tasks.workunit.client.0.smithi170.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-08T04:11:19.020 INFO:tasks.workunit.client.0.smithi170.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-08T04:11:19.060 INFO:tasks.workunit.client.0.smithi170.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-08T04:11:19.061 INFO:tasks.workunit.client.0.smithi170.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-08T04:11:19.061 INFO:tasks.workunit.client.0.smithi170.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-08T04:11:19.101 INFO:tasks.workunit.client.0.smithi170.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-08T04:11:19.104 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-05-08T04:11:19.104 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-08T04:11:19.161 INFO:tasks.workunit:Running workunits matching rados/test_libcephsqlite.sh on client.0... 2024-05-08T04:11:19.162 INFO:tasks.workunit:Running workunit rados/test_libcephsqlite.sh... 2024-05-08T04:11:19.162 DEBUG:teuthology.orchestra.run.smithi170:workunit test rados/test_libcephsqlite.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5ccf1c30cf724d5371f55439b3b3a90cb994fae2 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CEPH_ARGS='--id libcephsqlite --no-log-to-stderr' adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_libcephsqlite.sh cephsqlite 2024-05-08T04:11:19.220 INFO:tasks.workunit.client.0.smithi170.stderr:+ pool=cephsqlite 2024-05-08T04:11:19.220 INFO:tasks.workunit.client.0.smithi170.stderr:++ basename /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_libcephsqlite.sh 2024-05-08T04:11:19.221 INFO:tasks.workunit.client.0.smithi170.stderr:+ ns=test_libcephsqlite.sh 2024-05-08T04:11:19.221 INFO:tasks.workunit.client.0.smithi170.stderr:+ striper rm baz.db 2024-05-08T04:11:19.221 INFO:tasks.workunit.client.0.smithi170.stderr:+ rados --pool=cephsqlite --namespace=test_libcephsqlite.sh --striper rm baz.db 2024-05-08T04:11:19.266 INFO:tasks.workunit.client.0.smithi170.stderr:+ sqlite 2024-05-08T04:11:19.266 INFO:tasks.workunit.client.0.smithi170.stderr:+ background= 2024-05-08T04:11:19.266 INFO:tasks.workunit.client.0.smithi170.stderr:+ '[' '' = b ']' 2024-05-08T04:11:19.267 INFO:tasks.workunit.client.0.smithi170.stderr:++ cat 2024-05-08T04:11:19.267 INFO:tasks.workunit.client.0.smithi170.stderr:+ a='create table if not exists foo (a INT); 2024-05-08T04:11:19.268 INFO:tasks.workunit.client.0.smithi170.stderr:insert into foo (a) values (RANDOM()); 2024-05-08T04:11:19.268 INFO:tasks.workunit.client.0.smithi170.stderr:drop table foo;' 2024-05-08T04:11:19.268 INFO:tasks.workunit.client.0.smithi170.stderr:+ printf %s 'create table if not exists foo (a INT); 2024-05-08T04:11:19.268 INFO:tasks.workunit.client.0.smithi170.stderr:insert into foo (a) values (RANDOM()); 2024-05-08T04:11:19.268 INFO:tasks.workunit.client.0.smithi170.stderr:drop table foo;' 2024-05-08T04:11:19.268 INFO:tasks.workunit.client.0.smithi170.stderr:create table if not exists foo (a INT); 2024-05-08T04:11:19.268 INFO:tasks.workunit.client.0.smithi170.stderr:insert into foo (a) values (RANDOM()); 2024-05-08T04:11:19.268 INFO:tasks.workunit.client.0.smithi170.stderr:drop table foo;+ '[' '' '!=' b ']' 2024-05-08T04:11:19.268 INFO:tasks.workunit.client.0.smithi170.stderr:+ wait 2024-05-08T04:11:19.268 INFO:tasks.workunit.client.0.smithi170.stderr:+ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-05-08T04:11:19.440 INFO:tasks.workunit.client.0.smithi170.stderr: 2024-05-08T04:11:19.440 INFO:tasks.workunit.client.0.smithi170.stderr:real 0m0.174s 2024-05-08T04:11:19.440 INFO:tasks.workunit.client.0.smithi170.stderr:user 0m0.036s 2024-05-08T04:11:19.440 INFO:tasks.workunit.client.0.smithi170.stderr:sys 0m0.022s 2024-05-08T04:11:19.440 INFO:tasks.workunit.client.0.smithi170.stderr:+ striper stat baz.db 2024-05-08T04:11:19.440 INFO:tasks.workunit.client.0.smithi170.stderr:+ rados --pool=cephsqlite --namespace=test_libcephsqlite.sh --striper stat baz.db 2024-05-08T04:11:19.475 INFO:tasks.workunit.client.0.smithi170.stdout:cephsqlite/baz.db mtime 2024-05-08T04:11:19.000000+0000, size 8192 2024-05-08T04:11:19.477 INFO:tasks.workunit.client.0.smithi170.stderr:+ striper rm baz.db 2024-05-08T04:11:19.477 INFO:tasks.workunit.client.0.smithi170.stderr:+ rados --pool=cephsqlite --namespace=test_libcephsqlite.sh --striper rm baz.db 2024-05-08T04:11:19.521 INFO:tasks.workunit.client.0.smithi170.stderr:+ sqlite 2024-05-08T04:11:19.521 INFO:tasks.workunit.client.0.smithi170.stderr:++ repeat 10 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:19.521 INFO:tasks.workunit.client.0.smithi170.stderr:++ n=10 2024-05-08T04:11:19.521 INFO:tasks.workunit.client.0.smithi170.stderr:++ shift 2024-05-08T04:11:19.521 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i = 0 )) 2024-05-08T04:11:19.521 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 10 )) 2024-05-08T04:11:19.521 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:19.521 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:19.522 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 10 )) 2024-05-08T04:11:19.522 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:19.522 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:19.522 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 10 )) 2024-05-08T04:11:19.522 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:19.522 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:19.522 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 10 )) 2024-05-08T04:11:19.522 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:19.522 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:19.522 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 10 )) 2024-05-08T04:11:19.522 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:19.522 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:19.522 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 10 )) 2024-05-08T04:11:19.522 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:19.523 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:19.523 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 10 )) 2024-05-08T04:11:19.523 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:19.523 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:19.523 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 10 )) 2024-05-08T04:11:19.523 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:19.523 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:19.523 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 10 )) 2024-05-08T04:11:19.523 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:19.523 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:19.523 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 10 )) 2024-05-08T04:11:19.523 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:19.523 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:19.523 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 10 )) 2024-05-08T04:11:19.523 INFO:tasks.workunit.client.0.smithi170.stderr:+ background= 2024-05-08T04:11:19.524 INFO:tasks.workunit.client.0.smithi170.stderr:+ '[' '' = b ']' 2024-05-08T04:11:19.524 INFO:tasks.workunit.client.0.smithi170.stderr:++ cat 2024-05-08T04:11:19.524 INFO:tasks.workunit.client.0.smithi170.stderr:+ a='CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-05-08T04:11:19.524 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:19.524 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:19.524 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:19.524 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:19.524 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:19.524 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:19.524 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:19.524 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:19.524 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:19.524 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:19.524 INFO:tasks.workunit.client.0.smithi170.stderr:SELECT LENGTH(text) FROM rand; 2024-05-08T04:11:19.524 INFO:tasks.workunit.client.0.smithi170.stderr:DROP TABLE rand;' 2024-05-08T04:11:19.525 INFO:tasks.workunit.client.0.smithi170.stderr:+ printf %s 'CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-05-08T04:11:19.525 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:19.525 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:19.525 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:19.525 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:19.525 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:19.525 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:19.525 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:19.525 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:19.525 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:19.525 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:19.525 INFO:tasks.workunit.client.0.smithi170.stderr:SELECT LENGTH(text) FROM rand; 2024-05-08T04:11:19.525 INFO:tasks.workunit.client.0.smithi170.stderr:DROP TABLE rand;' 2024-05-08T04:11:19.525 INFO:tasks.workunit.client.0.smithi170.stderr:CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-05-08T04:11:19.525 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:19.525 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:19.526 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:19.526 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:19.526 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:19.526 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:19.526 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:19.526 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:19.526 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:19.526 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:19.526 INFO:tasks.workunit.client.0.smithi170.stderr:SELECT LENGTH(text) FROM rand; 2024-05-08T04:11:19.526 INFO:tasks.workunit.client.0.smithi170.stderr:DROP TABLE rand;+ '[' '' '!=' b ']' 2024-05-08T04:11:19.526 INFO:tasks.workunit.client.0.smithi170.stderr:+ wait 2024-05-08T04:11:19.527 INFO:tasks.workunit.client.0.smithi170.stderr:+ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-05-08T04:11:20.051 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.051 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.051 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.051 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.051 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.051 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.051 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.051 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.051 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.051 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.155 INFO:tasks.workunit.client.0.smithi170.stderr: 2024-05-08T04:11:20.156 INFO:tasks.workunit.client.0.smithi170.stderr:real 0m0.626s 2024-05-08T04:11:20.156 INFO:tasks.workunit.client.0.smithi170.stderr:user 0m0.079s 2024-05-08T04:11:20.156 INFO:tasks.workunit.client.0.smithi170.stderr:sys 0m0.042s 2024-05-08T04:11:20.156 INFO:tasks.workunit.client.0.smithi170.stderr:+ sqlite 2024-05-08T04:11:20.156 INFO:tasks.workunit.client.0.smithi170.stderr:++ repeat 100 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.156 INFO:tasks.workunit.client.0.smithi170.stderr:++ n=100 2024-05-08T04:11:20.156 INFO:tasks.workunit.client.0.smithi170.stderr:++ shift 2024-05-08T04:11:20.156 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i = 0 )) 2024-05-08T04:11:20.156 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.156 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.156 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.156 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.156 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.157 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.157 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.157 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.157 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.157 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.157 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.157 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.157 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.157 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.157 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.157 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.157 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.157 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.157 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.158 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.158 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.158 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.158 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.158 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.158 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.158 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.158 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.158 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.158 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.158 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.158 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.158 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.158 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.159 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.159 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.159 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.159 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.159 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.159 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.159 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.159 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.159 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.159 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.159 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.159 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.159 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.159 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.159 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.160 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.160 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.160 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.160 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.160 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.160 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.160 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.160 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.160 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.160 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.160 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.160 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.160 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.160 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.160 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.161 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.161 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.161 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.161 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.161 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.161 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.161 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.161 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.161 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.161 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.161 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.161 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.161 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.161 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.162 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.162 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.162 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.162 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.162 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.162 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.162 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.162 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.162 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.162 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.162 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.162 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.162 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.162 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.163 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.163 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.163 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.163 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.163 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.163 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.163 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.163 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.163 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.163 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.163 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.163 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.163 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.163 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.163 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.164 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.164 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.164 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.164 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.164 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.164 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.164 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.164 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.164 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.164 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.164 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.164 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.164 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.164 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.164 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.164 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.165 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.165 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.165 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.165 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.165 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.165 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.165 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.165 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.165 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.165 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.165 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.165 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.165 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.165 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.165 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.166 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.166 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.166 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.166 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.166 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.166 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.166 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.166 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.166 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.166 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.166 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.166 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.166 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.166 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.167 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.167 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.167 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.167 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.167 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.167 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.167 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.167 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.167 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.167 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.167 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.167 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.167 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.167 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.167 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.168 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.168 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.168 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.168 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.168 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.168 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.168 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.168 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.168 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.168 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.168 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.168 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.168 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.168 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.168 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.169 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.169 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.169 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.169 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.169 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.169 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.169 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.169 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.169 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.169 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.169 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.169 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.169 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.169 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.169 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.169 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.170 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.170 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.170 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.170 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.170 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.170 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.170 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.170 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.170 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.170 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.170 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.170 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.170 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.170 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.171 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.171 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.171 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.171 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.171 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.171 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.171 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.171 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.171 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.171 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.171 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.171 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.171 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.171 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.171 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.172 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.172 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.172 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.172 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.172 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.172 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.172 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.172 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.172 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.172 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.172 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.172 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.172 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.172 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.172 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.172 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.173 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.173 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.173 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.173 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.173 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.173 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.173 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.173 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.173 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.173 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.173 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.173 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.173 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.173 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.173 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.173 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.174 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.174 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.174 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.174 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.174 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.174 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.174 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.174 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.174 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.174 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.174 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.174 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.174 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.174 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.175 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.175 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.175 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.175 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.175 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.175 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.175 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.175 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.175 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.175 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.175 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.175 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.175 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.175 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.175 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.176 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.176 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.176 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.176 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.176 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.176 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.176 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.176 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.176 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.176 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.176 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.176 INFO:tasks.workunit.client.0.smithi170.stderr:++ echo 'INSERT INTO rand (text) VALUES (RANDOMBLOB(4096));' 2024-05-08T04:11:20.176 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( ++i )) 2024-05-08T04:11:20.176 INFO:tasks.workunit.client.0.smithi170.stderr:++ (( i < 100 )) 2024-05-08T04:11:20.176 INFO:tasks.workunit.client.0.smithi170.stderr:+ background= 2024-05-08T04:11:20.177 INFO:tasks.workunit.client.0.smithi170.stderr:+ '[' '' = b ']' 2024-05-08T04:11:20.177 INFO:tasks.workunit.client.0.smithi170.stderr:++ cat 2024-05-08T04:11:20.177 INFO:tasks.workunit.client.0.smithi170.stderr:+ a='BEGIN TRANSACTION; 2024-05-08T04:11:20.177 INFO:tasks.workunit.client.0.smithi170.stderr:CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-05-08T04:11:20.177 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.177 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.177 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.177 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.177 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.177 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.177 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.177 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.177 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.177 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.177 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.177 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.178 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.178 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.178 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.178 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.178 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.178 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.178 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.178 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.178 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.178 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.178 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.178 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.178 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.179 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.179 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.179 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.179 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.179 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.179 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.179 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.179 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.179 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.179 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.179 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.179 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.179 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.179 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.179 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.179 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.180 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.180 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.180 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.180 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.180 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.180 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.180 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.180 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.180 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.180 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.180 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.180 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.180 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.180 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.180 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.180 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.181 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.181 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.181 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.181 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.181 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.181 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.181 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.181 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.181 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.181 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.181 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.181 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.181 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.181 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.181 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.181 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.182 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.182 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.182 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.182 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.182 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.182 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.182 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.182 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.182 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.182 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.182 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.182 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.182 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.182 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.183 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.183 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.183 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.183 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.183 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.183 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.183 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.183 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.183 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.183 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.183 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.183 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.183 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.183 INFO:tasks.workunit.client.0.smithi170.stderr:COMMIT; 2024-05-08T04:11:20.183 INFO:tasks.workunit.client.0.smithi170.stderr:SELECT LENGTH(text) FROM rand; 2024-05-08T04:11:20.184 INFO:tasks.workunit.client.0.smithi170.stderr:DROP TABLE rand;' 2024-05-08T04:11:20.184 INFO:tasks.workunit.client.0.smithi170.stderr:+ printf %s 'BEGIN TRANSACTION; 2024-05-08T04:11:20.184 INFO:tasks.workunit.client.0.smithi170.stderr:CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-05-08T04:11:20.184 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.184 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.184 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.184 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.184 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.184 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.184 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.184 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.184 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.184 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.184 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.184 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.184 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.185 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.185 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.185 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.185 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.185 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.185 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.185 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.185 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.185 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.185 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.185 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.185 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.185 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.185 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.185 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.185 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.186 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.186 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.186 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.186 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.186 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.186 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.186 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.186 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.186 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.186 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.186 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.186 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.186 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.186 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.187 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.187 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.187 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.187 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.187 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.187 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.187 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.187 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.187 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.187 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.188 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.188 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.188 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.188 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.188 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.188 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.188 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.188 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.188 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.188 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.188 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.188 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.188 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.189 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.189 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.189 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.189 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.189 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.189 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.189 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.189 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.189 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.189 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.189 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.189 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.189 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.189 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.189 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.189 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.190 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.190 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.190 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.190 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.190 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.190 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.190 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.190 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.190 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.190 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.190 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.190 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.190 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.190 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.191 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.191 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.191 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.191 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.191 INFO:tasks.workunit.client.0.smithi170.stderr:COMMIT; 2024-05-08T04:11:20.191 INFO:tasks.workunit.client.0.smithi170.stderr:SELECT LENGTH(text) FROM rand; 2024-05-08T04:11:20.191 INFO:tasks.workunit.client.0.smithi170.stderr:DROP TABLE rand;' 2024-05-08T04:11:20.191 INFO:tasks.workunit.client.0.smithi170.stderr:BEGIN TRANSACTION; 2024-05-08T04:11:20.191 INFO:tasks.workunit.client.0.smithi170.stderr:CREATE TABLE IF NOT EXISTS rand(text BLOB NOT NULL); 2024-05-08T04:11:20.191 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.191 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.191 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.191 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.191 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.191 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.192 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.192 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.192 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.192 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.192 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.192 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.192 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.192 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.192 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.192 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.192 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.192 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.192 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.192 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.192 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.193 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.193 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.193 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.193 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.193 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.193 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.193 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.193 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.193 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.193 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.193 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.193 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.193 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.193 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.193 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.194 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.194 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.194 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.194 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.194 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.194 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.194 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.194 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.194 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.194 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.194 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.194 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.194 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.194 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.194 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.195 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.195 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.195 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.195 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.195 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.195 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.195 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.195 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.195 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.195 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.195 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.195 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.195 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.195 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.195 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.196 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.196 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.196 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.196 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.196 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.196 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.196 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.196 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.196 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.196 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.196 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.196 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.196 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.196 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.196 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.196 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.197 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.197 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.197 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.197 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.197 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.197 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.197 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.197 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.197 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.197 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.197 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.198 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.198 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.198 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.198 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.198 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.198 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.198 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO rand (text) VALUES (RANDOMBLOB(4096)); 2024-05-08T04:11:20.198 INFO:tasks.workunit.client.0.smithi170.stderr:COMMIT; 2024-05-08T04:11:20.198 INFO:tasks.workunit.client.0.smithi170.stderr:SELECT LENGTH(text) FROM rand; 2024-05-08T04:11:20.198 INFO:tasks.workunit.client.0.smithi170.stderr:DROP TABLE rand;+ '[' '' '!=' b ']' 2024-05-08T04:11:20.198 INFO:tasks.workunit.client.0.smithi170.stderr:+ wait 2024-05-08T04:11:20.198 INFO:tasks.workunit.client.0.smithi170.stderr:+ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-05-08T04:11:20.367 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.368 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.368 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.368 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.368 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.368 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.368 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.368 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.368 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.369 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.369 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.369 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.369 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.369 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.369 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.369 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.369 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.369 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.369 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.369 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.369 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.369 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.369 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.370 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.370 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.370 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.370 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.370 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.370 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.370 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.370 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.370 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.370 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.370 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.370 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.370 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.370 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.371 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.371 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.371 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.371 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.371 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.371 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.371 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.371 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.371 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.371 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.371 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.371 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.371 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.371 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.371 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.371 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.372 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.372 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.372 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.372 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.372 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.372 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.372 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.372 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.372 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.372 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.372 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.372 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.372 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.372 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.372 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.373 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.373 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.373 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.373 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.373 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.373 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.373 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.373 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.373 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.373 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.373 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.373 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.373 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.373 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.373 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.373 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.374 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.374 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.374 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.374 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.374 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.374 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.374 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.374 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.374 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.374 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.374 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.374 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.374 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.374 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.375 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.375 INFO:tasks.workunit.client.0.smithi170.stdout:4096 2024-05-08T04:11:20.892 INFO:tasks.workunit.client.0.smithi170.stderr: 2024-05-08T04:11:20.892 INFO:tasks.workunit.client.0.smithi170.stderr:real 0m0.747s 2024-05-08T04:11:20.892 INFO:tasks.workunit.client.0.smithi170.stderr:user 0m0.078s 2024-05-08T04:11:20.892 INFO:tasks.workunit.client.0.smithi170.stderr:sys 0m0.036s 2024-05-08T04:11:20.892 INFO:tasks.workunit.client.0.smithi170.stderr:+ striper rm baz.db 2024-05-08T04:11:20.892 INFO:tasks.workunit.client.0.smithi170.stderr:+ rados --pool=cephsqlite --namespace=test_libcephsqlite.sh --striper rm baz.db 2024-05-08T04:11:20.933 INFO:tasks.workunit.client.0.smithi170.stderr:+ date 2024-05-08T04:11:20.934 INFO:tasks.workunit.client.0.smithi170.stdout:Wed May 8 04:11:20 AM UTC 2024 2024-05-08T04:11:20.935 INFO:tasks.workunit.client.0.smithi170.stderr:+ sqlite b 2024-05-08T04:11:20.935 INFO:tasks.workunit.client.0.smithi170.stderr:+ background=b 2024-05-08T04:11:20.935 INFO:tasks.workunit.client.0.smithi170.stderr:+ '[' b = b ']' 2024-05-08T04:11:20.935 INFO:tasks.workunit.client.0.smithi170.stderr:+ shift 2024-05-08T04:11:20.935 INFO:tasks.workunit.client.0.smithi170.stderr:++ cat 2024-05-08T04:11:20.936 INFO:tasks.workunit.client.0.smithi170.stderr:+ a='CREATE TABLE foo (a BLOB); 2024-05-08T04:11:20.936 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO foo VALUES ("start"); 2024-05-08T04:11:20.936 INFO:tasks.workunit.client.0.smithi170.stderr:WITH RECURSIVE c(x) AS 2024-05-08T04:11:20.936 INFO:tasks.workunit.client.0.smithi170.stderr: ( 2024-05-08T04:11:20.936 INFO:tasks.workunit.client.0.smithi170.stderr: VALUES(1) 2024-05-08T04:11:20.936 INFO:tasks.workunit.client.0.smithi170.stderr: UNION ALL 2024-05-08T04:11:20.936 INFO:tasks.workunit.client.0.smithi170.stderr: SELECT x+1 2024-05-08T04:11:20.936 INFO:tasks.workunit.client.0.smithi170.stderr: FROM c 2024-05-08T04:11:20.936 INFO:tasks.workunit.client.0.smithi170.stderr: ) 2024-05-08T04:11:20.936 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO foo (a) 2024-05-08T04:11:20.936 INFO:tasks.workunit.client.0.smithi170.stderr: SELECT RANDOMBLOB(1<<20) 2024-05-08T04:11:20.936 INFO:tasks.workunit.client.0.smithi170.stderr: FROM c 2024-05-08T04:11:20.936 INFO:tasks.workunit.client.0.smithi170.stderr: LIMIT (1<<20);' 2024-05-08T04:11:20.937 INFO:tasks.workunit.client.0.smithi170.stderr:+ printf %s 'CREATE TABLE foo (a BLOB); 2024-05-08T04:11:20.937 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO foo VALUES ("start"); 2024-05-08T04:11:20.937 INFO:tasks.workunit.client.0.smithi170.stderr:WITH RECURSIVE c(x) AS 2024-05-08T04:11:20.937 INFO:tasks.workunit.client.0.smithi170.stderr: ( 2024-05-08T04:11:20.937 INFO:tasks.workunit.client.0.smithi170.stderr: VALUES(1) 2024-05-08T04:11:20.937 INFO:tasks.workunit.client.0.smithi170.stderr: UNION ALL 2024-05-08T04:11:20.937 INFO:tasks.workunit.client.0.smithi170.stderr: SELECT x+1 2024-05-08T04:11:20.937 INFO:tasks.workunit.client.0.smithi170.stderr: FROM c 2024-05-08T04:11:20.937 INFO:tasks.workunit.client.0.smithi170.stderr: ) 2024-05-08T04:11:20.937 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO foo (a) 2024-05-08T04:11:20.937 INFO:tasks.workunit.client.0.smithi170.stderr: SELECT RANDOMBLOB(1<<20) 2024-05-08T04:11:20.937 INFO:tasks.workunit.client.0.smithi170.stderr: FROM c 2024-05-08T04:11:20.937 INFO:tasks.workunit.client.0.smithi170.stderr: LIMIT (1<<20);' 2024-05-08T04:11:20.937 INFO:tasks.workunit.client.0.smithi170.stderr:CREATE TABLE foo (a BLOB); 2024-05-08T04:11:20.937 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO foo VALUES ("start"); 2024-05-08T04:11:20.937 INFO:tasks.workunit.client.0.smithi170.stderr:WITH RECURSIVE c(x) AS 2024-05-08T04:11:20.938 INFO:tasks.workunit.client.0.smithi170.stderr: ( 2024-05-08T04:11:20.938 INFO:tasks.workunit.client.0.smithi170.stderr: VALUES(1) 2024-05-08T04:11:20.938 INFO:tasks.workunit.client.0.smithi170.stderr: UNION ALL 2024-05-08T04:11:20.938 INFO:tasks.workunit.client.0.smithi170.stderr: SELECT x+1 2024-05-08T04:11:20.938 INFO:tasks.workunit.client.0.smithi170.stderr: FROM c 2024-05-08T04:11:20.938 INFO:tasks.workunit.client.0.smithi170.stderr: ) 2024-05-08T04:11:20.938 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO foo (a) 2024-05-08T04:11:20.938 INFO:tasks.workunit.client.0.smithi170.stderr: SELECT RANDOMBLOB(1<<20) 2024-05-08T04:11:20.938 INFO:tasks.workunit.client.0.smithi170.stderr: FROM c 2024-05-08T04:11:20.938 INFO:tasks.workunit.client.0.smithi170.stderr: LIMIT (1<<20);+ '[' b '!=' b ']' 2024-05-08T04:11:20.938 INFO:tasks.workunit.client.0.smithi170.stderr:+ sleep 10 2024-05-08T04:11:20.938 INFO:tasks.workunit.client.0.smithi170.stderr:+ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-05-08T04:11:30.937 INFO:tasks.workunit.client.0.smithi170.stdout:done 2024-05-08T04:11:30.937 INFO:tasks.workunit.client.0.smithi170.stderr:+ echo done 2024-05-08T04:11:30.937 INFO:tasks.workunit.client.0.smithi170.stderr:+ jobs -l 2024-05-08T04:11:30.938 INFO:tasks.workunit.client.0.smithi170.stdout:[1]+ 36107 Running sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd ".open file:///$pool:$ns/baz.db?vfs=ceph" -cmd '.output stdout' <<< "$a" & 2024-05-08T04:11:30.938 INFO:tasks.workunit.client.0.smithi170.stderr:++ jobs -p 2024-05-08T04:11:30.938 INFO:tasks.workunit.client.0.smithi170.stderr:+ kill -KILL -- 36107 2024-05-08T04:11:30.938 INFO:tasks.workunit.client.0.smithi170.stderr:+ date 2024-05-08T04:11:30.939 INFO:tasks.workunit.client.0.smithi170.stdout:Wed May 8 04:11:30 AM UTC 2024 2024-05-08T04:11:30.939 INFO:tasks.workunit.client.0.smithi170.stderr:+ wait 2024-05-08T04:11:30.939 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_libcephsqlite.sh: line 95: 36107 Killed sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd ".open file:///$pool:$ns/baz.db?vfs=ceph" -cmd '.output stdout' <<< "$a" 2024-05-08T04:11:30.939 INFO:tasks.workunit.client.0.smithi170.stderr:+ date 2024-05-08T04:11:30.940 INFO:tasks.workunit.client.0.smithi170.stdout:Wed May 8 04:11:30 AM UTC 2024 2024-05-08T04:11:30.940 INFO:tasks.workunit.client.0.smithi170.stderr:++ sqlite 2024-05-08T04:11:30.940 INFO:tasks.workunit.client.0.smithi170.stderr:++ background= 2024-05-08T04:11:30.940 INFO:tasks.workunit.client.0.smithi170.stderr:++ '[' '' = b ']' 2024-05-08T04:11:30.941 INFO:tasks.workunit.client.0.smithi170.stderr:+++ cat 2024-05-08T04:11:30.941 INFO:tasks.workunit.client.0.smithi170.stderr:++ a='SELECT COUNT(*) FROM foo;' 2024-05-08T04:11:30.941 INFO:tasks.workunit.client.0.smithi170.stderr:++ printf %s 'SELECT COUNT(*) FROM foo;' 2024-05-08T04:11:30.942 INFO:tasks.workunit.client.0.smithi170.stderr:SELECT COUNT(*) FROM foo;++ '[' '' '!=' b ']' 2024-05-08T04:11:30.942 INFO:tasks.workunit.client.0.smithi170.stderr:++ wait 2024-05-08T04:11:30.942 INFO:tasks.workunit.client.0.smithi170.stderr:++ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-05-08T04:11:58.202 INFO:tasks.workunit.client.0.smithi170.stderr:+ n=1 2024-05-08T04:11:58.202 INFO:tasks.workunit.client.0.smithi170.stderr:+ '[' 1 -eq 1 ']' 2024-05-08T04:11:58.202 INFO:tasks.workunit.client.0.smithi170.stderr:+ striper rm baz.db 2024-05-08T04:11:58.202 INFO:tasks.workunit.client.0.smithi170.stderr:+ rados --pool=cephsqlite --namespace=test_libcephsqlite.sh --striper rm baz.db 2024-05-08T04:11:58.258 INFO:tasks.workunit.client.0.smithi170.stderr:+ date 2024-05-08T04:11:58.259 INFO:tasks.workunit.client.0.smithi170.stdout:Wed May 8 04:11:58 AM UTC 2024 2024-05-08T04:11:58.259 INFO:tasks.workunit.client.0.smithi170.stderr:+ sqlite b 2024-05-08T04:11:58.259 INFO:tasks.workunit.client.0.smithi170.stderr:+ background=b 2024-05-08T04:11:58.259 INFO:tasks.workunit.client.0.smithi170.stderr:+ '[' b = b ']' 2024-05-08T04:11:58.260 INFO:tasks.workunit.client.0.smithi170.stderr:+ shift 2024-05-08T04:11:58.260 INFO:tasks.workunit.client.0.smithi170.stderr:++ cat 2024-05-08T04:11:58.260 INFO:tasks.workunit.client.0.smithi170.stderr:+ a='CREATE TABLE foo (a BLOB); 2024-05-08T04:11:58.260 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO foo VALUES ("start"); 2024-05-08T04:11:58.261 INFO:tasks.workunit.client.0.smithi170.stderr:WITH RECURSIVE c(x) AS 2024-05-08T04:11:58.261 INFO:tasks.workunit.client.0.smithi170.stderr: ( 2024-05-08T04:11:58.261 INFO:tasks.workunit.client.0.smithi170.stderr: VALUES(1) 2024-05-08T04:11:58.261 INFO:tasks.workunit.client.0.smithi170.stderr: UNION ALL 2024-05-08T04:11:58.261 INFO:tasks.workunit.client.0.smithi170.stderr: SELECT x+1 2024-05-08T04:11:58.261 INFO:tasks.workunit.client.0.smithi170.stderr: FROM c 2024-05-08T04:11:58.261 INFO:tasks.workunit.client.0.smithi170.stderr: ) 2024-05-08T04:11:58.261 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO foo (a) 2024-05-08T04:11:58.261 INFO:tasks.workunit.client.0.smithi170.stderr: SELECT RANDOMBLOB(1<<20) 2024-05-08T04:11:58.261 INFO:tasks.workunit.client.0.smithi170.stderr: FROM c 2024-05-08T04:11:58.261 INFO:tasks.workunit.client.0.smithi170.stderr: LIMIT (1<<20);' 2024-05-08T04:11:58.261 INFO:tasks.workunit.client.0.smithi170.stderr:+ printf %s 'CREATE TABLE foo (a BLOB); 2024-05-08T04:11:58.261 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO foo VALUES ("start"); 2024-05-08T04:11:58.261 INFO:tasks.workunit.client.0.smithi170.stderr:WITH RECURSIVE c(x) AS 2024-05-08T04:11:58.262 INFO:tasks.workunit.client.0.smithi170.stderr: ( 2024-05-08T04:11:58.262 INFO:tasks.workunit.client.0.smithi170.stderr: VALUES(1) 2024-05-08T04:11:58.262 INFO:tasks.workunit.client.0.smithi170.stderr: UNION ALL 2024-05-08T04:11:58.262 INFO:tasks.workunit.client.0.smithi170.stderr: SELECT x+1 2024-05-08T04:11:58.262 INFO:tasks.workunit.client.0.smithi170.stderr: FROM c 2024-05-08T04:11:58.262 INFO:tasks.workunit.client.0.smithi170.stderr: ) 2024-05-08T04:11:58.262 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO foo (a) 2024-05-08T04:11:58.262 INFO:tasks.workunit.client.0.smithi170.stderr: SELECT RANDOMBLOB(1<<20) 2024-05-08T04:11:58.262 INFO:tasks.workunit.client.0.smithi170.stderr: FROM c 2024-05-08T04:11:58.262 INFO:tasks.workunit.client.0.smithi170.stderr: LIMIT (1<<20);' 2024-05-08T04:11:58.262 INFO:tasks.workunit.client.0.smithi170.stderr:CREATE TABLE foo (a BLOB); 2024-05-08T04:11:58.262 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO foo VALUES ("start"); 2024-05-08T04:11:58.262 INFO:tasks.workunit.client.0.smithi170.stderr:WITH RECURSIVE c(x) AS 2024-05-08T04:11:58.263 INFO:tasks.workunit.client.0.smithi170.stderr: ( 2024-05-08T04:11:58.263 INFO:tasks.workunit.client.0.smithi170.stderr: VALUES(1) 2024-05-08T04:11:58.263 INFO:tasks.workunit.client.0.smithi170.stderr: UNION ALL 2024-05-08T04:11:58.263 INFO:tasks.workunit.client.0.smithi170.stderr: SELECT x+1 2024-05-08T04:11:58.263 INFO:tasks.workunit.client.0.smithi170.stderr: FROM c 2024-05-08T04:11:58.263 INFO:tasks.workunit.client.0.smithi170.stderr: ) 2024-05-08T04:11:58.263 INFO:tasks.workunit.client.0.smithi170.stderr:INSERT INTO foo (a) 2024-05-08T04:11:58.263 INFO:tasks.workunit.client.0.smithi170.stderr: SELECT RANDOMBLOB(1<<20) 2024-05-08T04:11:58.263 INFO:tasks.workunit.client.0.smithi170.stderr: FROM c 2024-05-08T04:11:58.263 INFO:tasks.workunit.client.0.smithi170.stderr: LIMIT (1<<20);+ '[' b '!=' b ']' 2024-05-08T04:11:58.263 INFO:tasks.workunit.client.0.smithi170.stderr:+ sleep 20 2024-05-08T04:11:58.263 INFO:tasks.workunit.client.0.smithi170.stderr:+ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-05-08T04:12:18.262 INFO:tasks.workunit.client.0.smithi170.stderr:+ jobs -l 2024-05-08T04:12:18.262 INFO:tasks.workunit.client.0.smithi170.stdout:[1]+ 36175 Running sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd ".open file:///$pool:$ns/baz.db?vfs=ceph" -cmd '.output stdout' <<< "$a" & 2024-05-08T04:12:18.263 INFO:tasks.workunit.client.0.smithi170.stderr:++ jobs -p 2024-05-08T04:12:18.263 INFO:tasks.workunit.client.0.smithi170.stderr:+ kill -STOP -- 36175 2024-05-08T04:12:18.263 INFO:tasks.workunit.client.0.smithi170.stderr:+ sleep 45 2024-05-08T04:13:03.264 INFO:tasks.workunit.client.0.smithi170.stderr:+ date 2024-05-08T04:13:03.265 INFO:tasks.workunit.client.0.smithi170.stdout:Wed May 8 04:13:03 AM UTC 2024 2024-05-08T04:13:03.266 INFO:tasks.workunit.client.0.smithi170.stderr:++ jobs -p 2024-05-08T04:13:03.266 INFO:tasks.workunit.client.0.smithi170.stderr:+ kill -CONT -- 36175 2024-05-08T04:13:03.266 INFO:tasks.workunit.client.0.smithi170.stderr:+ sleep 10 2024-05-08T04:13:03.286 INFO:tasks.workunit.client.0.smithi170.stderr:Error: near line 3: disk I/O error 2024-05-08T04:13:13.267 INFO:tasks.workunit.client.0.smithi170.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_libcephsqlite.sh: line 129: 36175 Segmentation fault (core dumped) sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd ".open file:///$pool:$ns/baz.db?vfs=ceph" -cmd '.output stdout' <<< "$a" 2024-05-08T04:13:13.267 INFO:tasks.workunit.client.0.smithi170.stderr:+ date 2024-05-08T04:13:13.268 INFO:tasks.workunit.client.0.smithi170.stdout:Wed May 8 04:13:13 AM UTC 2024 2024-05-08T04:13:13.268 INFO:tasks.workunit.client.0.smithi170.stderr:+ wait 2024-05-08T04:13:13.269 INFO:tasks.workunit.client.0.smithi170.stderr:+ date 2024-05-08T04:13:13.269 INFO:tasks.workunit.client.0.smithi170.stdout:Wed May 8 04:13:13 AM UTC 2024 2024-05-08T04:13:13.269 INFO:tasks.workunit.client.0.smithi170.stderr:++ sqlite 2024-05-08T04:13:13.270 INFO:tasks.workunit.client.0.smithi170.stderr:++ background= 2024-05-08T04:13:13.270 INFO:tasks.workunit.client.0.smithi170.stderr:++ '[' '' = b ']' 2024-05-08T04:13:13.270 INFO:tasks.workunit.client.0.smithi170.stderr:+++ cat 2024-05-08T04:13:13.271 INFO:tasks.workunit.client.0.smithi170.stderr:++ a='SELECT COUNT(*) FROM foo;' 2024-05-08T04:13:13.271 INFO:tasks.workunit.client.0.smithi170.stderr:++ printf %s 'SELECT COUNT(*) FROM foo;' 2024-05-08T04:13:13.271 INFO:tasks.workunit.client.0.smithi170.stderr:SELECT COUNT(*) FROM foo;++ '[' '' '!=' b ']' 2024-05-08T04:13:13.271 INFO:tasks.workunit.client.0.smithi170.stderr:++ wait 2024-05-08T04:13:13.271 INFO:tasks.workunit.client.0.smithi170.stderr:++ sqlite3 -cmd '.output /dev/null' -cmd '.load libcephsqlite.so' -cmd 'pragma journal_mode = PERSIST' -cmd '.open file:///cephsqlite:test_libcephsqlite.sh/baz.db?vfs=ceph' -cmd '.output stdout' 2024-05-08T04:13:14.319 INFO:tasks.workunit.client.0.smithi170.stderr:+ n=1 2024-05-08T04:13:14.320 INFO:tasks.workunit.client.0.smithi170.stderr:+ '[' 1 -eq 1 ']' 2024-05-08T04:13:14.320 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-08T04:13:14.320 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-08T04:13:14.388 INFO:tasks.workunit:Stopping ['rados/test_libcephsqlite.sh cephsqlite'] on client.0... 2024-05-08T04:13:14.388 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-08T04:13:14.698 DEBUG:teuthology.parallel:result is None 2024-05-08T04:13:14.699 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-08T04:13:14.723 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-08T04:13:14.723 DEBUG:teuthology.orchestra.run.smithi170:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-05-08T04:13:14.783 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-05-08T04:13:14.783 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-08T04:13:14.793 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T04:13:14.793 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:13:14.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.920+0000 7f5e83fa5640 1 Processor -- start 2024-05-08T04:13:14.922 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.920+0000 7f5e83fa5640 1 -- start start 2024-05-08T04:13:14.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.921+0000 7f5e83fa5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5e7c0b8380 0x7f5e7c0aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:14.923 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.921+0000 7f5e83fa5640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e7c0b8760 con 0x7f5e7c140f90 2024-05-08T04:13:14.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.921+0000 7f5e83fa5640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e7c0ac800 con 0x7f5e7c145680 2024-05-08T04:13:14.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.921+0000 7f5e83fa5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f5e7c0ab920 con 0x7f5e7c0b8380 2024-05-08T04:13:14.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.921+0000 7f5e81d41640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5e7c0b8380 0x7f5e7c0aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:14.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.921+0000 7f5e82542640 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f5e7c140f90 0x7f5e7c141420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:52464/0 (socket says 172.21.15.170:52464) 2024-05-08T04:13:14.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.921+0000 7f5e82542640 1 -- 172.21.15.170:0/2335767865 learned_addr learned my addr 172.21.15.170:0/2335767865 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-05-08T04:13:14.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.922+0000 7f5e81d41640 1 -- 172.21.15.170:0/2335767865 >> v1:172.21.15.187:6789/0 conn(0x7f5e7c145680 legacy=0x7f5e7c145ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:13:14.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.922+0000 7f5e81d41640 1 -- 172.21.15.170:0/2335767865 >> v1:172.21.15.170:6789/0 conn(0x7f5e7c140f90 legacy=0x7f5e7c141420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:14.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.922+0000 7f5e80d3f640 1 -- 172.21.15.170:0/2335767865 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3826247679 0 0) 0x7f5e7c0b8760 con 0x7f5e7c140f90 2024-05-08T04:13:14.924 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.922+0000 7f5e81d41640 1 -- 172.21.15.170:0/2335767865 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e7c0ab410 con 0x7f5e7c0b8380 2024-05-08T04:13:14.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.922+0000 7f5e81d41640 1 --2- 172.21.15.170:0/2335767865 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5e7c0b8380 0x7f5e7c0aaed0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5e78009dc0 tx=0x7f5e7802bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e60082683ee4a81f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:14.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.922+0000 7f5e80d3f640 1 -- 172.21.15.170:0/2335767865 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e78039070 con 0x7f5e7c0b8380 2024-05-08T04:13:14.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.922+0000 7f5e80d3f640 1 -- 172.21.15.170:0/2335767865 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e7802cb90 con 0x7f5e7c0b8380 2024-05-08T04:13:14.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.922+0000 7f5e80d3f640 1 -- 172.21.15.170:0/2335767865 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e78034440 con 0x7f5e7c0b8380 2024-05-08T04:13:14.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.923+0000 7f5e83fa5640 1 -- 172.21.15.170:0/2335767865 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5e7c0b8380 msgr2=0x7f5e7c0aaed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:14.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.923+0000 7f5e83fa5640 1 --2- 172.21.15.170:0/2335767865 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5e7c0b8380 0x7f5e7c0aaed0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5e78009dc0 tx=0x7f5e7802bde0 comp rx=0 tx=0).stop 2024-05-08T04:13:14.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.923+0000 7f5e83fa5640 1 -- 172.21.15.170:0/2335767865 shutdown_connections 2024-05-08T04:13:14.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.923+0000 7f5e83fa5640 1 --2- 172.21.15.170:0/2335767865 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5e7c0b8380 0x7f5e7c0aaed0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:14.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.923+0000 7f5e83fa5640 1 -- 172.21.15.170:0/2335767865 >> 172.21.15.170:0/2335767865 conn(0x7f5e7c06df80 msgr2=0x7f5e7c0b7990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:14.926 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.923+0000 7f5e83fa5640 1 -- 172.21.15.170:0/2335767865 shutdown_connections 2024-05-08T04:13:14.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.923+0000 7f5e83fa5640 1 -- 172.21.15.170:0/2335767865 wait complete. 2024-05-08T04:13:14.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.924+0000 7f5e83fa5640 1 Processor -- start 2024-05-08T04:13:14.927 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.924+0000 7f5e83fa5640 1 -- start start 2024-05-08T04:13:14.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.925+0000 7f5e83fa5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e7c0b8380 0x7f5e7c1344e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:14.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.925+0000 7f5e83fa5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e7c140f90 0x7f5e7c139270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:14.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.925+0000 7f5e83fa5640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5e7c145680 0x7f5e7c1385a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:14.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.925+0000 7f5e83fa5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7c14d5d0 con 0x7f5e7c140f90 2024-05-08T04:13:14.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.925+0000 7f5e83fa5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5e7c141ec0 con 0x7f5e7c0b8380 2024-05-08T04:13:14.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.925+0000 7f5e83fa5640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f5e7c14d750 con 0x7f5e7c145680 2024-05-08T04:13:14.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.925+0000 7f5e82542640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5e7c145680 0x7f5e7c1385a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:14.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.925+0000 7f5e82542640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5e7c145680 0x7f5e7c1385a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:55728/0 (socket says 172.21.15.170:55728) 2024-05-08T04:13:14.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.925+0000 7f5e81540640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e7c140f90 0x7f5e7c139270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:14.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.925+0000 7f5e82542640 1 -- 172.21.15.170:0/1969011916 learned_addr learned my addr 172.21.15.170:0/1969011916 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:13:14.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.925+0000 7f5e81d41640 1 --2- 172.21.15.170:0/1969011916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e7c0b8380 0x7f5e7c1344e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:14.930 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.925+0000 7f5e82542640 1 -- 172.21.15.170:0/1969011916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e7c0b8380 msgr2=0x7f5e7c1344e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:14.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.925+0000 7f5e82542640 1 --2- 172.21.15.170:0/1969011916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e7c0b8380 0x7f5e7c1344e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:14.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.925+0000 7f5e82542640 1 -- 172.21.15.170:0/1969011916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e7c140f90 msgr2=0x7f5e7c139270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:14.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.925+0000 7f5e82542640 1 --2- 172.21.15.170:0/1969011916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e7c140f90 0x7f5e7c139270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:14.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.925+0000 7f5e82542640 1 -- 172.21.15.170:0/1969011916 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e7c138ae0 con 0x7f5e7c145680 2024-05-08T04:13:14.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.925+0000 7f5e81540640 1 --2- 172.21.15.170:0/1969011916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e7c140f90 0x7f5e7c139270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:13:14.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.925+0000 7f5e82542640 1 --2- 172.21.15.170:0/1969011916 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5e7c145680 0x7f5e7c1385a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5e7400c410 tx=0x7f5e7400c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:14.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.925+0000 7f5e727fc640 1 -- 172.21.15.170:0/1969011916 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e740074e0 con 0x7f5e7c145680 2024-05-08T04:13:14.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.925+0000 7f5e727fc640 1 -- 172.21.15.170:0/1969011916 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e740047e0 con 0x7f5e7c145680 2024-05-08T04:13:14.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.925+0000 7f5e727fc640 1 -- 172.21.15.170:0/1969011916 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e74004bb0 con 0x7f5e7c145680 2024-05-08T04:13:14.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.925+0000 7f5e83fa5640 1 -- 172.21.15.170:0/1969011916 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e7c1305d0 con 0x7f5e7c145680 2024-05-08T04:13:14.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.925+0000 7f5e83fa5640 1 -- 172.21.15.170:0/1969011916 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e7c130d40 con 0x7f5e7c145680 2024-05-08T04:13:14.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.926+0000 7f5e83fa5640 1 -- 172.21.15.170:0/1969011916 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e7c06e9e0 con 0x7f5e7c145680 2024-05-08T04:13:14.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.927+0000 7f5e727fc640 1 -- 172.21.15.170:0/1969011916 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f5e74011070 con 0x7f5e7c145680 2024-05-08T04:13:14.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.928+0000 7f5e727fc640 1 --2- 172.21.15.170:0/1969011916 >> v2:172.21.15.187:6816/101007897 conn(0x7f5e60076290 0x7f5e60078750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:14.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.928+0000 7f5e727fc640 1 -- 172.21.15.170:0/1969011916 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 4525+0+0 (secure 0 0 0) 0x7f5e74096b20 con 0x7f5e7c145680 2024-05-08T04:13:14.931 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.928+0000 7f5e81d41640 1 --2- 172.21.15.170:0/1969011916 >> v2:172.21.15.187:6816/101007897 conn(0x7f5e60076290 0x7f5e60078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:14.932 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.928+0000 7f5e81d41640 1 --2- 172.21.15.170:0/1969011916 >> v2:172.21.15.187:6816/101007897 conn(0x7f5e60076290 0x7f5e60078750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5e78036000 tx=0x7f5e78030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:14.934 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:14.932+0000 7f5e727fc640 1 -- 172.21.15.170:0/1969011916 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5e74060360 con 0x7f5e7c145680 2024-05-08T04:13:15.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.110+0000 7f5e83fa5640 1 -- 172.21.15.170:0/1969011916 --> v2:172.21.15.187:6816/101007897 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5e7c06b400 con 0x7f5e60076290 2024-05-08T04:13:15.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.114+0000 7f5e727fc640 1 -- 172.21.15.170:0/1969011916 <== mgr.4107 v2:172.21.15.187:6816/101007897 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+117676 (secure 0 0 0) 0x7f5e7c06b400 con 0x7f5e60076290 2024-05-08T04:13:15.116 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:13:15.117 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-05-08T04:13:15.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.116+0000 7f5e83fa5640 1 -- 172.21.15.170:0/1969011916 >> v2:172.21.15.187:6816/101007897 conn(0x7f5e60076290 msgr2=0x7f5e60078750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:15.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.116+0000 7f5e83fa5640 1 --2- 172.21.15.170:0/1969011916 >> v2:172.21.15.187:6816/101007897 conn(0x7f5e60076290 0x7f5e60078750 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5e78036000 tx=0x7f5e78030000 comp rx=0 tx=0).stop 2024-05-08T04:13:15.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.116+0000 7f5e83fa5640 1 -- 172.21.15.170:0/1969011916 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5e7c145680 msgr2=0x7f5e7c1385a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:15.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.116+0000 7f5e83fa5640 1 --2- 172.21.15.170:0/1969011916 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5e7c145680 0x7f5e7c1385a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5e7400c410 tx=0x7f5e7400c8d0 comp rx=0 tx=0).stop 2024-05-08T04:13:15.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.116+0000 7f5e83fa5640 1 -- 172.21.15.170:0/1969011916 shutdown_connections 2024-05-08T04:13:15.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.116+0000 7f5e83fa5640 1 --2- 172.21.15.170:0/1969011916 >> v2:172.21.15.187:6816/101007897 conn(0x7f5e60076290 0x7f5e60078750 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:15.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.116+0000 7f5e83fa5640 1 --2- 172.21.15.170:0/1969011916 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5e7c145680 0x7f5e7c1385a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:15.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.116+0000 7f5e83fa5640 1 --2- 172.21.15.170:0/1969011916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e7c140f90 0x7f5e7c139270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:15.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.116+0000 7f5e83fa5640 1 --2- 172.21.15.170:0/1969011916 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5e7c0b8380 0x7f5e7c1344e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:15.119 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.116+0000 7f5e83fa5640 1 -- 172.21.15.170:0/1969011916 >> 172.21.15.170:0/1969011916 conn(0x7f5e7c06df80 msgr2=0x7f5e7c0b4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:15.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.116+0000 7f5e83fa5640 1 -- 172.21.15.170:0/1969011916 shutdown_connections 2024-05-08T04:13:15.120 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.116+0000 7f5e83fa5640 1 -- 172.21.15.170:0/1969011916 wait complete. 2024-05-08T04:13:15.129 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2024-05-08T04:13:14.241795+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3581805864,"num_objects":883,"num_object_clones":0,"num_object_copies":1766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":883,"num_whiteouts":0,"num_read":2570,"num_read_kb":2570,"num_write":164609,"num_write_kb":5657393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167289,"ondisk_log_size":167289,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9338120,"kb_used_data":6665288,"kb_used_omap":12,"kb_used_meta":2672755,"kb_avail":745636600,"statfs":{"total":773094113280,"available":763531878400,"internally_reserved":0,"allocated":6825254912,"data_stored":6862685808,"data_compressed":194295728,"data_compressed_allocated":202113024,"data_compressed_original":244916224,"omap_allocated":12751,"internal_metadata":2736901681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20480,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.525470"},"pg_stats":[{"pgid":"3.1f","version":"21'3951","reported_seq":3984,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.165552+0000","last_change":"2024-05-08T04:08:27.732101+0000","last_active":"2024-05-08T04:11:58.165552+0000","last_peered":"2024-05-08T04:11:58.165552+0000","last_clean":"2024-05-08T04:11:58.165552+0000","last_became_active":"2024-05-08T04:08:27.731781+0000","last_became_peered":"2024-05-08T04:08:27.731781+0000","last_unstale":"2024-05-08T04:11:58.165552+0000","last_undegraded":"2024-05-08T04:11:58.165552+0000","last_fullsized":"2024-05-08T04:11:58.165552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":3951,"log_dups_size":0,"ondisk_log_size":3951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:46:39.944802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":62,"num_read_kb":52,"num_write":3964,"num_write_kb":169350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"22'7380","reported_seq":7398,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:12:05.363320+0000","last_change":"2024-05-08T04:08:27.731984+0000","last_active":"2024-05-08T04:12:05.363320+0000","last_peered":"2024-05-08T04:12:05.363320+0000","last_clean":"2024-05-08T04:12:05.363320+0000","last_became_active":"2024-05-08T04:08:27.731598+0000","last_became_peered":"2024-05-08T04:08:27.731598+0000","last_unstale":"2024-05-08T04:12:05.363320+0000","last_undegraded":"2024-05-08T04:12:05.363320+0000","last_fullsized":"2024-05-08T04:12:05.363320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":7380,"log_dups_size":0,"ondisk_log_size":7380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:41:22.212562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":91594752,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":7365,"num_write_kb":213614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"22'5764","reported_seq":5793,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:12:10.049692+0000","last_change":"2024-05-08T04:08:27.732284+0000","last_active":"2024-05-08T04:12:10.049692+0000","last_peered":"2024-05-08T04:12:10.049692+0000","last_clean":"2024-05-08T04:12:10.049692+0000","last_became_active":"2024-05-08T04:08:27.732006+0000","last_became_peered":"2024-05-08T04:08:27.732006+0000","last_unstale":"2024-05-08T04:12:10.049692+0000","last_undegraded":"2024-05-08T04:12:10.049692+0000","last_fullsized":"2024-05-08T04:12:10.049692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":5764,"log_dups_size":0,"ondisk_log_size":5764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:24:55.281049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":56,"num_read_kb":44,"num_write":5770,"num_write_kb":177284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"21'4466","reported_seq":4488,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.166652+0000","last_change":"2024-05-08T04:08:27.733083+0000","last_active":"2024-05-08T04:11:58.166652+0000","last_peered":"2024-05-08T04:11:58.166652+0000","last_clean":"2024-05-08T04:11:58.166652+0000","last_became_active":"2024-05-08T04:08:27.732710+0000","last_became_peered":"2024-05-08T04:08:27.732710+0000","last_unstale":"2024-05-08T04:11:58.166652+0000","last_undegraded":"2024-05-08T04:11:58.166652+0000","last_fullsized":"2024-05-08T04:11:58.166652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":4466,"log_dups_size":0,"ondisk_log_size":4466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:40:43.219199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125829120,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":4481,"num_write_kb":186066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"22'4173","reported_seq":4190,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.397733+0000","last_change":"2024-05-08T04:08:27.726522+0000","last_active":"2024-05-08T04:11:58.397733+0000","last_peered":"2024-05-08T04:11:58.397733+0000","last_clean":"2024-05-08T04:11:58.397733+0000","last_became_active":"2024-05-08T04:08:27.726232+0000","last_became_peered":"2024-05-08T04:08:27.726232+0000","last_unstale":"2024-05-08T04:11:58.397733+0000","last_undegraded":"2024-05-08T04:11:58.397733+0000","last_fullsized":"2024-05-08T04:11:58.397733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":4173,"log_dups_size":0,"ondisk_log_size":4173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:07.708671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120203616,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":4136,"num_write_kb":200298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"22'6140","reported_seq":6158,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:12:03.565212+0000","last_change":"2024-05-08T04:08:27.722514+0000","last_active":"2024-05-08T04:12:03.565212+0000","last_peered":"2024-05-08T04:12:03.565212+0000","last_clean":"2024-05-08T04:12:03.565212+0000","last_became_active":"2024-05-08T04:08:27.721701+0000","last_became_peered":"2024-05-08T04:08:27.721701+0000","last_unstale":"2024-05-08T04:12:03.565212+0000","last_undegraded":"2024-05-08T04:12:03.565212+0000","last_fullsized":"2024-05-08T04:12:03.565212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":6140,"log_dups_size":0,"ondisk_log_size":6140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:44.554641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":6123,"num_write_kb":209934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"22'5185","reported_seq":5203,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:12:14.567740+0000","last_change":"2024-05-08T04:08:27.728824+0000","last_active":"2024-05-08T04:12:14.567740+0000","last_peered":"2024-05-08T04:12:14.567740+0000","last_clean":"2024-05-08T04:12:14.567740+0000","last_became_active":"2024-05-08T04:08:27.728564+0000","last_became_peered":"2024-05-08T04:08:27.728564+0000","last_unstale":"2024-05-08T04:12:14.567740+0000","last_undegraded":"2024-05-08T04:12:14.567740+0000","last_fullsized":"2024-05-08T04:12:14.567740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":5185,"log_dups_size":0,"ondisk_log_size":5185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:55.969304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142606336,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":5146,"num_write_kb":193669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"22'7560","reported_seq":7579,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:12:14.637052+0000","last_change":"2024-05-08T04:08:27.728565+0000","last_active":"2024-05-08T04:12:14.637052+0000","last_peered":"2024-05-08T04:12:14.637052+0000","last_clean":"2024-05-08T04:12:14.637052+0000","last_became_active":"2024-05-08T04:08:27.728205+0000","last_became_peered":"2024-05-08T04:08:27.728205+0000","last_unstale":"2024-05-08T04:12:14.637052+0000","last_undegraded":"2024-05-08T04:12:14.637052+0000","last_fullsized":"2024-05-08T04:12:14.637052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":7560,"log_dups_size":0,"ondisk_log_size":7560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:22:53.989163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142606336,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":12,"num_read_kb":135,"num_write":7569,"num_write_kb":191194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"22'5903","reported_seq":5918,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:12:17.354833+0000","last_change":"2024-05-08T04:08:27.732104+0000","last_active":"2024-05-08T04:12:17.354833+0000","last_peered":"2024-05-08T04:12:17.354833+0000","last_clean":"2024-05-08T04:12:17.354833+0000","last_became_active":"2024-05-08T04:08:27.731769+0000","last_became_peered":"2024-05-08T04:08:27.731769+0000","last_unstale":"2024-05-08T04:12:17.354833+0000","last_undegraded":"2024-05-08T04:12:17.354833+0000","last_fullsized":"2024-05-08T04:12:17.354833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":5903,"log_dups_size":0,"ondisk_log_size":5903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:21:26.194969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5896,"num_write_kb":160193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"22'4607","reported_seq":4623,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.338176+0000","last_change":"2024-05-08T04:08:27.731045+0000","last_active":"2024-05-08T04:11:58.338176+0000","last_peered":"2024-05-08T04:11:58.338176+0000","last_clean":"2024-05-08T04:11:58.338176+0000","last_became_active":"2024-05-08T04:08:27.730749+0000","last_became_peered":"2024-05-08T04:08:27.730749+0000","last_unstale":"2024-05-08T04:11:58.338176+0000","last_undegraded":"2024-05-08T04:11:58.338176+0000","last_fullsized":"2024-05-08T04:11:58.338176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":4607,"log_dups_size":0,"ondisk_log_size":4607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:17:07.247072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":127271424,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":4592,"num_write_kb":174343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"22'7467","reported_seq":7793,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:12:13.235858+0000","last_change":"2024-05-08T04:08:27.729609+0000","last_active":"2024-05-08T04:12:13.235858+0000","last_peered":"2024-05-08T04:12:13.235858+0000","last_clean":"2024-05-08T04:12:13.235858+0000","last_became_active":"2024-05-08T04:08:27.729318+0000","last_became_peered":"2024-05-08T04:08:27.729318+0000","last_unstale":"2024-05-08T04:12:13.235858+0000","last_undegraded":"2024-05-08T04:12:13.235858+0000","last_fullsized":"2024-05-08T04:12:13.235858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":7467,"log_dups_size":0,"ondisk_log_size":7467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:17:45.553221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112068432,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1045,"num_read_kb":996,"num_write":7685,"num_write_kb":175598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"22'6085","reported_seq":6101,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:12:07.645890+0000","last_change":"2024-05-08T04:08:27.730319+0000","last_active":"2024-05-08T04:12:07.645890+0000","last_peered":"2024-05-08T04:12:07.645890+0000","last_clean":"2024-05-08T04:12:07.645890+0000","last_became_active":"2024-05-08T04:08:27.729781+0000","last_became_peered":"2024-05-08T04:08:27.729781+0000","last_unstale":"2024-05-08T04:12:07.645890+0000","last_undegraded":"2024-05-08T04:12:07.645890+0000","last_fullsized":"2024-05-08T04:12:07.645890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":6085,"log_dups_size":0,"ondisk_log_size":6085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:29:26.256991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123143848,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6035,"num_write_kb":178933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.167611+0000","last_change":"2024-05-08T04:08:21.697423+0000","last_active":"2024-05-08T04:11:58.167611+0000","last_peered":"2024-05-08T04:11:58.167611+0000","last_clean":"2024-05-08T04:11:58.167611+0000","last_became_active":"2024-05-08T04:08:21.697149+0000","last_became_peered":"2024-05-08T04:08:21.697149+0000","last_unstale":"2024-05-08T04:11:58.167611+0000","last_undegraded":"2024-05-08T04:11:58.167611+0000","last_fullsized":"2024-05-08T04:11:58.167611+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:59:38.354372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"22'7339","reported_seq":7378,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:12:19.316202+0000","last_change":"2024-05-08T04:08:27.725473+0000","last_active":"2024-05-08T04:12:19.316202+0000","last_peered":"2024-05-08T04:12:19.316202+0000","last_clean":"2024-05-08T04:12:19.316202+0000","last_became_active":"2024-05-08T04:08:27.725165+0000","last_became_peered":"2024-05-08T04:08:27.725165+0000","last_unstale":"2024-05-08T04:12:19.316202+0000","last_undegraded":"2024-05-08T04:12:19.316202+0000","last_fullsized":"2024-05-08T04:12:19.316202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":7339,"log_dups_size":0,"ondisk_log_size":7339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:06:19.688474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142738448,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":74,"num_read_kb":59,"num_write":7371,"num_write_kb":187653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.137646+0000","last_change":"2024-05-08T04:08:21.696942+0000","last_active":"2024-05-08T04:11:58.137646+0000","last_peered":"2024-05-08T04:11:58.137646+0000","last_clean":"2024-05-08T04:11:58.137646+0000","last_became_active":"2024-05-08T04:08:21.696632+0000","last_became_peered":"2024-05-08T04:08:21.696632+0000","last_unstale":"2024-05-08T04:11:58.137646+0000","last_undegraded":"2024-05-08T04:11:58.137646+0000","last_fullsized":"2024-05-08T04:11:58.137646+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:23:25.156383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"22'3710","reported_seq":3726,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.394698+0000","last_change":"2024-05-08T04:08:27.730834+0000","last_active":"2024-05-08T04:11:58.394698+0000","last_peered":"2024-05-08T04:11:58.394698+0000","last_clean":"2024-05-08T04:11:58.394698+0000","last_became_active":"2024-05-08T04:08:27.730530+0000","last_became_peered":"2024-05-08T04:08:27.730530+0000","last_unstale":"2024-05-08T04:11:58.394698+0000","last_undegraded":"2024-05-08T04:11:58.394698+0000","last_fullsized":"2024-05-08T04:11:58.394698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":3710,"log_dups_size":0,"ondisk_log_size":3710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:51:02.242727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3661,"num_write_kb":168963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.138336+0000","last_change":"2024-05-08T04:08:21.696814+0000","last_active":"2024-05-08T04:11:58.138336+0000","last_peered":"2024-05-08T04:11:58.138336+0000","last_clean":"2024-05-08T04:11:58.138336+0000","last_became_active":"2024-05-08T04:08:21.696403+0000","last_became_peered":"2024-05-08T04:08:21.696403+0000","last_unstale":"2024-05-08T04:11:58.138336+0000","last_undegraded":"2024-05-08T04:11:58.138336+0000","last_fullsized":"2024-05-08T04:11:58.138336+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:02:04.075137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"22'6934","reported_seq":6950,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:12:16.551309+0000","last_change":"2024-05-08T04:08:27.721621+0000","last_active":"2024-05-08T04:12:16.551309+0000","last_peered":"2024-05-08T04:12:16.551309+0000","last_clean":"2024-05-08T04:12:16.551309+0000","last_became_active":"2024-05-08T04:08:27.721303+0000","last_became_peered":"2024-05-08T04:08:27.721303+0000","last_unstale":"2024-05-08T04:12:16.551309+0000","last_undegraded":"2024-05-08T04:12:16.551309+0000","last_fullsized":"2024-05-08T04:12:16.551309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":6934,"log_dups_size":0,"ondisk_log_size":6934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:59:37.780694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6930,"num_write_kb":193544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.137534+0000","last_change":"2024-05-08T04:08:21.693307+0000","last_active":"2024-05-08T04:11:58.137534+0000","last_peered":"2024-05-08T04:11:58.137534+0000","last_clean":"2024-05-08T04:11:58.137534+0000","last_became_active":"2024-05-08T04:08:21.693014+0000","last_became_peered":"2024-05-08T04:08:21.693014+0000","last_unstale":"2024-05-08T04:11:58.137534+0000","last_undegraded":"2024-05-08T04:11:58.137534+0000","last_fullsized":"2024-05-08T04:11:58.137534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:56.218680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"22'4514","reported_seq":4531,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:12:18.455836+0000","last_change":"2024-05-08T04:08:27.729798+0000","last_active":"2024-05-08T04:12:18.455836+0000","last_peered":"2024-05-08T04:12:18.455836+0000","last_clean":"2024-05-08T04:12:18.455836+0000","last_became_active":"2024-05-08T04:08:27.729504+0000","last_became_peered":"2024-05-08T04:08:27.729504+0000","last_unstale":"2024-05-08T04:12:18.455836+0000","last_undegraded":"2024-05-08T04:12:18.455836+0000","last_fullsized":"2024-05-08T04:12:18.455836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":4514,"log_dups_size":0,"ondisk_log_size":4514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:58:47.867425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96535048,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":4515,"num_write_kb":160844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"21'6272","reported_seq":6401,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.140568+0000","last_change":"2024-05-08T04:08:27.726831+0000","last_active":"2024-05-08T04:11:58.140568+0000","last_peered":"2024-05-08T04:11:58.140568+0000","last_clean":"2024-05-08T04:11:58.140568+0000","last_became_active":"2024-05-08T04:08:27.726527+0000","last_became_peered":"2024-05-08T04:08:27.726527+0000","last_unstale":"2024-05-08T04:11:58.140568+0000","last_undegraded":"2024-05-08T04:11:58.140568+0000","last_fullsized":"2024-05-08T04:11:58.140568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":6272,"log_dups_size":0,"ondisk_log_size":6272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:26:39.243499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":126223376,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":153,"num_read_kb":139,"num_write":6308,"num_write_kb":247140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.166594+0000","last_change":"2024-05-08T04:08:21.699221+0000","last_active":"2024-05-08T04:11:58.166594+0000","last_peered":"2024-05-08T04:11:58.166594+0000","last_clean":"2024-05-08T04:11:58.166594+0000","last_became_active":"2024-05-08T04:08:21.699013+0000","last_became_peered":"2024-05-08T04:08:21.699013+0000","last_unstale":"2024-05-08T04:11:58.166594+0000","last_undegraded":"2024-05-08T04:11:58.166594+0000","last_fullsized":"2024-05-08T04:11:58.166594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:23:08.040183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"22'3067","reported_seq":3083,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.391790+0000","last_change":"2024-05-08T04:08:27.723409+0000","last_active":"2024-05-08T04:11:58.391790+0000","last_peered":"2024-05-08T04:11:58.391790+0000","last_clean":"2024-05-08T04:11:58.391790+0000","last_became_active":"2024-05-08T04:08:27.723118+0000","last_became_peered":"2024-05-08T04:08:27.723118+0000","last_unstale":"2024-05-08T04:11:58.391790+0000","last_undegraded":"2024-05-08T04:11:58.391790+0000","last_fullsized":"2024-05-08T04:11:58.391790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":3067,"log_dups_size":0,"ondisk_log_size":3067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:31:49.593896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3019,"num_write_kb":152195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.137838+0000","last_change":"2024-05-08T04:08:21.799283+0000","last_active":"2024-05-08T04:11:58.137838+0000","last_peered":"2024-05-08T04:11:58.137838+0000","last_clean":"2024-05-08T04:11:58.137838+0000","last_became_active":"2024-05-08T04:08:21.798995+0000","last_became_peered":"2024-05-08T04:08:21.798995+0000","last_unstale":"2024-05-08T04:11:58.137838+0000","last_undegraded":"2024-05-08T04:11:58.137838+0000","last_fullsized":"2024-05-08T04:11:58.137838+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:37:16.795888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"22'3228","reported_seq":3245,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.397567+0000","last_change":"2024-05-08T04:08:27.726742+0000","last_active":"2024-05-08T04:11:58.397567+0000","last_peered":"2024-05-08T04:11:58.397567+0000","last_clean":"2024-05-08T04:11:58.397567+0000","last_became_active":"2024-05-08T04:08:27.726457+0000","last_became_peered":"2024-05-08T04:08:27.726457+0000","last_unstale":"2024-05-08T04:11:58.397567+0000","last_undegraded":"2024-05-08T04:11:58.397567+0000","last_fullsized":"2024-05-08T04:11:58.397567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":3228,"log_dups_size":0,"ondisk_log_size":3228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:36:59.353164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104923656,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":3211,"num_write_kb":154644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.137996+0000","last_change":"2024-05-08T04:08:21.697070+0000","last_active":"2024-05-08T04:11:58.137996+0000","last_peered":"2024-05-08T04:11:58.137996+0000","last_clean":"2024-05-08T04:11:58.137996+0000","last_became_active":"2024-05-08T04:08:21.696370+0000","last_became_peered":"2024-05-08T04:08:21.696370+0000","last_unstale":"2024-05-08T04:11:58.137996+0000","last_undegraded":"2024-05-08T04:11:58.137996+0000","last_fullsized":"2024-05-08T04:11:58.137996+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:54.981822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"22'5074","reported_seq":5090,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:12:11.590538+0000","last_change":"2024-05-08T04:08:27.732812+0000","last_active":"2024-05-08T04:12:11.590538+0000","last_peered":"2024-05-08T04:12:11.590538+0000","last_clean":"2024-05-08T04:12:11.590538+0000","last_became_active":"2024-05-08T04:08:27.732446+0000","last_became_peered":"2024-05-08T04:08:27.732446+0000","last_unstale":"2024-05-08T04:12:11.590538+0000","last_undegraded":"2024-05-08T04:12:11.590538+0000","last_fullsized":"2024-05-08T04:12:11.590538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":5074,"log_dups_size":0,"ondisk_log_size":5074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:34:38.276980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5070,"num_write_kb":160386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.166095+0000","last_change":"2024-05-08T04:08:21.696942+0000","last_active":"2024-05-08T04:11:58.166095+0000","last_peered":"2024-05-08T04:11:58.166095+0000","last_clean":"2024-05-08T04:11:58.166095+0000","last_became_active":"2024-05-08T04:08:21.696659+0000","last_became_peered":"2024-05-08T04:08:21.696659+0000","last_unstale":"2024-05-08T04:11:58.166095+0000","last_undegraded":"2024-05-08T04:11:58.166095+0000","last_fullsized":"2024-05-08T04:11:58.166095+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:50:44.209520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":114,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.137889+0000","last_change":"2024-05-08T04:08:20.691470+0000","last_active":"2024-05-08T04:11:58.137889+0000","last_peered":"2024-05-08T04:11:58.137889+0000","last_clean":"2024-05-08T04:11:58.137889+0000","last_became_active":"2024-05-08T04:08:20.691204+0000","last_became_peered":"2024-05-08T04:08:20.691204+0000","last_unstale":"2024-05-08T04:11:58.137889+0000","last_undegraded":"2024-05-08T04:11:58.137889+0000","last_fullsized":"2024-05-08T04:11:58.137889+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:19.679313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:19.679313+0000","last_clean_scrub_stamp":"2024-05-08T04:08:19.679313+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:52:31.185659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"22'3937","reported_seq":3958,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.392812+0000","last_change":"2024-05-08T04:08:27.729757+0000","last_active":"2024-05-08T04:11:58.392812+0000","last_peered":"2024-05-08T04:11:58.392812+0000","last_clean":"2024-05-08T04:11:58.392812+0000","last_became_active":"2024-05-08T04:08:27.729321+0000","last_became_peered":"2024-05-08T04:08:27.729321+0000","last_unstale":"2024-05-08T04:11:58.392812+0000","last_undegraded":"2024-05-08T04:11:58.392812+0000","last_fullsized":"2024-05-08T04:11:58.392812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":3937,"log_dups_size":0,"ondisk_log_size":3937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:12:28.413005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109182976,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":3879,"num_write_kb":165315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"22'9381","reported_seq":9518,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:12:49.841474+0000","last_change":"2024-05-08T04:08:27.721874+0000","last_active":"2024-05-08T04:12:49.841474+0000","last_peered":"2024-05-08T04:12:49.841474+0000","last_clean":"2024-05-08T04:12:49.841474+0000","last_became_active":"2024-05-08T04:08:27.721534+0000","last_became_peered":"2024-05-08T04:08:27.721534+0000","last_unstale":"2024-05-08T04:12:49.841474+0000","last_undegraded":"2024-05-08T04:12:49.841474+0000","last_fullsized":"2024-05-08T04:12:49.841474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":9381,"log_dups_size":0,"ondisk_log_size":9381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:59:46.563205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":139724624,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":382,"num_read_kb":338,"num_write":6452,"num_write_kb":194667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"22'3827","reported_seq":3865,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:12:16.074345+0000","last_change":"2024-05-08T04:08:27.726164+0000","last_active":"2024-05-08T04:12:16.074345+0000","last_peered":"2024-05-08T04:12:16.074345+0000","last_clean":"2024-05-08T04:12:16.074345+0000","last_became_active":"2024-05-08T04:08:27.725833+0000","last_became_peered":"2024-05-08T04:08:27.725833+0000","last_unstale":"2024-05-08T04:12:16.074345+0000","last_undegraded":"2024-05-08T04:12:16.074345+0000","last_fullsized":"2024-05-08T04:12:16.074345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":3827,"log_dups_size":0,"ondisk_log_size":3827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:35.399549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":85458944,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":91,"num_read_kb":137,"num_write":3837,"num_write_kb":155137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"21'4041","reported_seq":4081,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.165639+0000","last_change":"2024-05-08T04:08:27.727947+0000","last_active":"2024-05-08T04:11:58.165639+0000","last_peered":"2024-05-08T04:11:58.165639+0000","last_clean":"2024-05-08T04:11:58.165639+0000","last_became_active":"2024-05-08T04:08:27.727653+0000","last_became_peered":"2024-05-08T04:08:27.727653+0000","last_unstale":"2024-05-08T04:11:58.165639+0000","last_undegraded":"2024-05-08T04:11:58.165639+0000","last_fullsized":"2024-05-08T04:11:58.165639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":4041,"log_dups_size":0,"ondisk_log_size":4041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:14:37.705194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130023424,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":90,"num_read_kb":141,"num_write":4071,"num_write_kb":155757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"22'5322","reported_seq":5346,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.392917+0000","last_change":"2024-05-08T04:08:27.730956+0000","last_active":"2024-05-08T04:11:58.392917+0000","last_peered":"2024-05-08T04:11:58.392917+0000","last_clean":"2024-05-08T04:11:58.392917+0000","last_became_active":"2024-05-08T04:08:27.730569+0000","last_became_peered":"2024-05-08T04:08:27.730569+0000","last_unstale":"2024-05-08T04:11:58.392917+0000","last_undegraded":"2024-05-08T04:11:58.392917+0000","last_fullsized":"2024-05-08T04:11:58.392917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":5322,"log_dups_size":0,"ondisk_log_size":5322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:05:40.969984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":5289,"num_write_kb":148492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"22'4023","reported_seq":4053,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.390778+0000","last_change":"2024-05-08T04:08:27.733357+0000","last_active":"2024-05-08T04:11:58.390778+0000","last_peered":"2024-05-08T04:11:58.390778+0000","last_clean":"2024-05-08T04:11:58.390778+0000","last_became_active":"2024-05-08T04:08:27.733070+0000","last_became_peered":"2024-05-08T04:08:27.733070+0000","last_unstale":"2024-05-08T04:11:58.390778+0000","last_undegraded":"2024-05-08T04:11:58.390778+0000","last_fullsized":"2024-05-08T04:11:58.390778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":4023,"log_dups_size":0,"ondisk_log_size":4023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:33:08.617696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":103481344,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":53,"num_read_kb":105,"num_write":4128,"num_write_kb":186691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"22'3219","reported_seq":3313,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.434132+0000","last_change":"2024-05-08T04:08:27.730671+0000","last_active":"2024-05-08T04:11:58.434132+0000","last_peered":"2024-05-08T04:11:58.434132+0000","last_clean":"2024-05-08T04:11:58.434132+0000","last_became_active":"2024-05-08T04:08:27.730372+0000","last_became_peered":"2024-05-08T04:08:27.730372+0000","last_unstale":"2024-05-08T04:11:58.434132+0000","last_undegraded":"2024-05-08T04:11:58.434132+0000","last_fullsized":"2024-05-08T04:11:58.434132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":3219,"log_dups_size":0,"ondisk_log_size":3219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:55:59.085956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":78389776,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":301,"num_read_kb":226,"num_write":3489,"num_write_kb":113772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"22'4347","reported_seq":4366,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:13:03.293177+0000","last_change":"2024-05-08T04:08:27.729948+0000","last_active":"2024-05-08T04:13:03.293177+0000","last_peered":"2024-05-08T04:13:03.293177+0000","last_clean":"2024-05-08T04:13:03.293177+0000","last_became_active":"2024-05-08T04:08:27.729584+0000","last_became_peered":"2024-05-08T04:08:27.729584+0000","last_unstale":"2024-05-08T04:13:03.293177+0000","last_undegraded":"2024-05-08T04:13:03.293177+0000","last_fullsized":"2024-05-08T04:13:03.293177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":4347,"log_dups_size":0,"ondisk_log_size":4347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:35:18.017771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95482384,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":4309,"num_write_kb":147779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"22'5304","reported_seq":5328,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.392364+0000","last_change":"2024-05-08T04:08:27.727021+0000","last_active":"2024-05-08T04:11:58.392364+0000","last_peered":"2024-05-08T04:11:58.392364+0000","last_clean":"2024-05-08T04:11:58.392364+0000","last_became_active":"2024-05-08T04:08:27.726692+0000","last_became_peered":"2024-05-08T04:08:27.726692+0000","last_unstale":"2024-05-08T04:11:58.392364+0000","last_undegraded":"2024-05-08T04:11:58.392364+0000","last_fullsized":"2024-05-08T04:11:58.392364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":5304,"log_dups_size":0,"ondisk_log_size":5304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:16:39.270048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120979456,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":5269,"num_write_kb":245378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"21'2450","reported_seq":2466,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.138003+0000","last_change":"2024-05-08T04:08:27.727459+0000","last_active":"2024-05-08T04:11:58.138003+0000","last_peered":"2024-05-08T04:11:58.138003+0000","last_clean":"2024-05-08T04:11:58.138003+0000","last_became_active":"2024-05-08T04:08:27.727138+0000","last_became_peered":"2024-05-08T04:08:27.727138+0000","last_unstale":"2024-05-08T04:11:58.138003+0000","last_undegraded":"2024-05-08T04:11:58.138003+0000","last_fullsized":"2024-05-08T04:11:58.138003+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":2450,"log_dups_size":0,"ondisk_log_size":2450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:32:43.624016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2442,"num_write_kb":115332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"22'3788","reported_seq":3812,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.400216+0000","last_change":"2024-05-08T04:08:27.729355+0000","last_active":"2024-05-08T04:11:58.400216+0000","last_peered":"2024-05-08T04:11:58.400216+0000","last_clean":"2024-05-08T04:11:58.400216+0000","last_became_active":"2024-05-08T04:08:27.729024+0000","last_became_peered":"2024-05-08T04:08:27.729024+0000","last_unstale":"2024-05-08T04:11:58.400216+0000","last_undegraded":"2024-05-08T04:11:58.400216+0000","last_fullsized":"2024-05-08T04:11:58.400216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":3788,"log_dups_size":0,"ondisk_log_size":3788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:28:16.629952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96600064,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":3752,"num_write_kb":156932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"22'8759","reported_seq":8777,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:12:10.448806+0000","last_change":"2024-05-08T04:08:27.729279+0000","last_active":"2024-05-08T04:12:10.448806+0000","last_peered":"2024-05-08T04:12:10.448806+0000","last_clean":"2024-05-08T04:12:10.448806+0000","last_became_active":"2024-05-08T04:08:27.728925+0000","last_became_peered":"2024-05-08T04:08:27.728925+0000","last_unstale":"2024-05-08T04:12:10.448806+0000","last_undegraded":"2024-05-08T04:12:10.448806+0000","last_fullsized":"2024-05-08T04:12:10.448806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":8759,"log_dups_size":0,"ondisk_log_size":8759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:40:38.725967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163709456,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":8744,"num_write_kb":214668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3581215496,"num_objects":881,"num_object_clones":0,"num_object_copies":1762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":881,"num_whiteouts":0,"num_read":2504,"num_read_kb":2515,"num_write":164508,"num_write_kb":5655765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6823510016,"data_stored":6861080728,"data_compressed":194285506,"data_compressed_allocated":202047488,"data_compressed_original":243892224,"omap_allocated":0,"internal_metadata":0},"log_size":167217,"ondisk_log_size":167217,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":10222,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542205,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1406956,"kb_used_data":1010724,"kb_used_omap":1,"kb_used_meta":396222,"kb_avail":92964884,"statfs":{"total":96636764160,"available":95196041216,"internally_reserved":0,"allocated":1034981376,"data_stored":1042910983,"data_compressed":31421929,"data_compressed_allocated":32739328,"data_compressed_original":41480192,"omap_allocated":1589,"internal_metadata":405731787},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542205,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406896,"kb_used_data":259176,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":93964944,"statfs":{"total":96636764160,"available":96220102656,"internally_reserved":0,"allocated":265396224,"data_stored":266322235,"data_compressed":7220440,"data_compressed_allocated":7479296,"data_compressed_original":8728576,"omap_allocated":1594,"internal_metadata":151255494},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542205,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1269668,"kb_used_data":891484,"kb_used_omap":1,"kb_used_meta":378174,"kb_avail":93102172,"statfs":{"total":96636764160,"available":95336624128,"internally_reserved":0,"allocated":912879616,"data_stored":917907315,"data_compressed":26020264,"data_compressed_allocated":26992640,"data_compressed_original":32751616,"omap_allocated":1595,"internal_metadata":387250629},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542205,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1904424,"kb_used_data":1372192,"kb_used_omap":1,"kb_used_meta":532222,"kb_avail":92467416,"statfs":{"total":96636764160,"available":94686633984,"internally_reserved":0,"allocated":1405124608,"data_stored":1412898687,"data_compressed":41362870,"data_compressed_allocated":43028480,"data_compressed_original":51744768,"omap_allocated":1595,"internal_metadata":544995781},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542205,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1508028,"kb_used_data":1053492,"kb_used_omap":1,"kb_used_meta":454526,"kb_avail":92863812,"statfs":{"total":96636764160,"available":95092543488,"internally_reserved":0,"allocated":1078775808,"data_stored":1080567039,"data_compressed":21885706,"data_compressed_allocated":22720512,"data_compressed_original":25346048,"omap_allocated":1593,"internal_metadata":465435079},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542205,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":952204,"kb_used_data":673988,"kb_used_omap":1,"kb_used_meta":278206,"kb_avail":93419636,"statfs":{"total":96636764160,"available":95661707264,"internally_reserved":0,"allocated":690163712,"data_stored":694964323,"data_compressed":20980979,"data_compressed_allocated":21929984,"data_compressed_original":27267072,"omap_allocated":1595,"internal_metadata":284883397},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542205,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1461540,"kb_used_data":1079196,"kb_used_omap":1,"kb_used_meta":382334,"kb_avail":92910300,"statfs":{"total":96636764160,"available":95140147200,"internally_reserved":0,"allocated":1105096704,"data_stored":1112874703,"data_compressed":36587350,"data_compressed_allocated":38072320,"data_compressed_original":46706688,"omap_allocated":1595,"internal_metadata":391510469},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574910,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":428404,"kb_used_data":325036,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":93943436,"statfs":{"total":96636764160,"available":96198078464,"internally_reserved":0,"allocated":332836864,"data_stored":334240523,"data_compressed":8816190,"data_compressed_allocated":9150464,"data_compressed_original":10891264,"omap_allocated":1595,"internal_metadata":105839045},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5111,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5111,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1104838656,"data_stored":1112277372,"data_compressed":36582239,"data_compressed_allocated":38039552,"data_compressed_original":46194688,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":689958912,"data_stored":694895920,"data_compressed":20980979,"data_compressed_allocated":21929984,"data_compressed_original":27267072,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":332632064,"data_stored":334172120,"data_compressed":8816190,"data_compressed_allocated":9150464,"data_compressed_original":10891264,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1078571008,"data_stored":1080498636,"data_compressed":21885706,"data_compressed_allocated":22720512,"data_compressed_original":25346048,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1404919808,"data_stored":1412830284,"data_compressed":41362870,"data_compressed_allocated":43028480,"data_compressed_original":51744768,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":912674816,"data_stored":917838912,"data_compressed":26020264,"data_compressed_allocated":26992640,"data_compressed_original":32751616,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":265191424,"data_stored":266253832,"data_compressed":7220440,"data_compressed_allocated":7479296,"data_compressed_original":8728576,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1034723328,"data_stored":1042313652,"data_compressed":31416818,"data_compressed_allocated":32706560,"data_compressed_original":40968192,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:13:15.131 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:13:15.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.228+0000 7f8ef9bab640 1 Processor -- start 2024-05-08T04:13:15.230 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.228+0000 7f8ef9bab640 1 -- start start 2024-05-08T04:13:15.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.229+0000 7f8ef9bab640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8ef406f670 0x7f8ef4143ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:15.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.229+0000 7f8ef9bab640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8ef406fa50 0x7f8ef413a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:15.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.229+0000 7f8ef9bab640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ef40aec40 con 0x7f8ef4072dc0 2024-05-08T04:13:15.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.229+0000 7f8ef9bab640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f8ef41447d0 con 0x7f8ef406fa50 2024-05-08T04:13:15.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.229+0000 7f8ef9bab640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f8ef4070370 con 0x7f8ef406f670 2024-05-08T04:13:15.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.229+0000 7f8ef37fe640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8ef406f670 0x7f8ef4143ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:15.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.229+0000 7f8ef3fff640 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8ef4072dc0 0x7f8ef4073270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:52480/0 (socket says 172.21.15.170:52480) 2024-05-08T04:13:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.229+0000 7f8ef3fff640 1 -- 172.21.15.170:0/2565135754 learned_addr learned my addr 172.21.15.170:0/2565135754 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-05-08T04:13:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.229+0000 7f8ef2ffd640 1 --2- 172.21.15.170:0/2565135754 >> v2:172.21.15.187:3300/0 conn(0x7f8ef406fa50 0x7f8ef413a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.229+0000 7f8ef27fc640 1 -- 172.21.15.170:0/2565135754 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2176431654 0 0) 0x7f8ef40aec40 con 0x7f8ef4072dc0 2024-05-08T04:13:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.229+0000 7f8ef37fe640 1 -- 172.21.15.170:0/2565135754 >> v2:172.21.15.187:3300/0 conn(0x7f8ef406fa50 msgr2=0x7f8ef413a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:13:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.229+0000 7f8ef37fe640 1 --2- 172.21.15.170:0/2565135754 >> v2:172.21.15.187:3300/0 conn(0x7f8ef406fa50 0x7f8ef413a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.229+0000 7f8ef37fe640 1 -- 172.21.15.170:0/2565135754 >> v1:172.21.15.170:6789/0 conn(0x7f8ef4072dc0 legacy=0x7f8ef4073270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.229+0000 7f8ef37fe640 1 -- 172.21.15.170:0/2565135754 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef4144420 con 0x7f8ef406f670 2024-05-08T04:13:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.230+0000 7f8ef37fe640 1 --2- 172.21.15.170:0/2565135754 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8ef406f670 0x7f8ef4143ee0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4009dc0 tx=0x7f8ee402be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a23ef7271eb2b10 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.230+0000 7f8ef27fc640 1 -- 172.21.15.170:0/2565135754 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee4039070 con 0x7f8ef406f670 2024-05-08T04:13:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.230+0000 7f8ef27fc640 1 -- 172.21.15.170:0/2565135754 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ee402cbb0 con 0x7f8ef406f670 2024-05-08T04:13:15.233 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.230+0000 7f8ef27fc640 1 -- 172.21.15.170:0/2565135754 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee4034440 con 0x7f8ef406f670 2024-05-08T04:13:15.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.231+0000 7f8ef9bab640 1 -- 172.21.15.170:0/2565135754 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8ef406f670 msgr2=0x7f8ef4143ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:15.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.231+0000 7f8ef9bab640 1 --2- 172.21.15.170:0/2565135754 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8ef406f670 0x7f8ef4143ee0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4009dc0 tx=0x7f8ee402be00 comp rx=0 tx=0).stop 2024-05-08T04:13:15.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.231+0000 7f8ef9bab640 1 -- 172.21.15.170:0/2565135754 shutdown_connections 2024-05-08T04:13:15.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.231+0000 7f8ef9bab640 1 --2- 172.21.15.170:0/2565135754 >> v2:172.21.15.187:3300/0 conn(0x7f8ef406fa50 0x7f8ef413a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:15.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.231+0000 7f8ef9bab640 1 --2- 172.21.15.170:0/2565135754 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8ef406f670 0x7f8ef4143ee0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:15.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.231+0000 7f8ef9bab640 1 -- 172.21.15.170:0/2565135754 >> 172.21.15.170:0/2565135754 conn(0x7f8ef40b4490 msgr2=0x7f8ef40b48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:15.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.231+0000 7f8ef9bab640 1 -- 172.21.15.170:0/2565135754 shutdown_connections 2024-05-08T04:13:15.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.231+0000 7f8ef9bab640 1 -- 172.21.15.170:0/2565135754 wait complete. 2024-05-08T04:13:15.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.232+0000 7f8ef9bab640 1 Processor -- start 2024-05-08T04:13:15.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.232+0000 7f8ef9bab640 1 -- start start 2024-05-08T04:13:15.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.233+0000 7f8ef9bab640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ef406f670 0x7f8ef41dcd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:15.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.233+0000 7f8ef9bab640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ef406fa50 0x7f8ef4132a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:15.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.233+0000 7f8ef9bab640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8ef4072dc0 0x7f8ef4132fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:15.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.233+0000 7f8ef9bab640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef4144de0 con 0x7f8ef406fa50 2024-05-08T04:13:15.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.233+0000 7f8ef9bab640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef4144c60 con 0x7f8ef406f670 2024-05-08T04:13:15.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.233+0000 7f8ef9bab640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f8ef414e7f0 con 0x7f8ef4072dc0 2024-05-08T04:13:15.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.233+0000 7f8ef2ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ef406fa50 0x7f8ef4132a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:15.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.233+0000 7f8ef3fff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8ef4072dc0 0x7f8ef4132fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:15.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.233+0000 7f8ef2ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ef406fa50 0x7f8ef4132a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36238/0 (socket says 172.21.15.170:36238) 2024-05-08T04:13:15.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.233+0000 7f8ef2ffd640 1 -- 172.21.15.170:0/3063088796 learned_addr learned my addr 172.21.15.170:0/3063088796 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:13:15.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.233+0000 7f8ef37fe640 1 --2- 172.21.15.170:0/3063088796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ef406f670 0x7f8ef41dcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:15.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.233+0000 7f8ef2ffd640 1 -- 172.21.15.170:0/3063088796 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8ef4072dc0 msgr2=0x7f8ef4132fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:15.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.233+0000 7f8ef2ffd640 1 --2- 172.21.15.170:0/3063088796 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8ef4072dc0 0x7f8ef4132fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:15.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.233+0000 7f8ef2ffd640 1 -- 172.21.15.170:0/3063088796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ef406f670 msgr2=0x7f8ef41dcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:15.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.233+0000 7f8ef2ffd640 1 --2- 172.21.15.170:0/3063088796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ef406f670 0x7f8ef41dcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:15.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.233+0000 7f8ef2ffd640 1 -- 172.21.15.170:0/3063088796 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef4143790 con 0x7f8ef406fa50 2024-05-08T04:13:15.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.233+0000 7f8ef37fe640 1 --2- 172.21.15.170:0/3063088796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ef406f670 0x7f8ef41dcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:13:15.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.233+0000 7f8ef3fff640 1 --2- 172.21.15.170:0/3063088796 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8ef4072dc0 0x7f8ef4132fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:13:15.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.233+0000 7f8ef2ffd640 1 --2- 172.21.15.170:0/3063088796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ef406fa50 0x7f8ef4132a60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800d800 tx=0x7f8ee800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:15.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.233+0000 7f8edbfff640 1 -- 172.21.15.170:0/3063088796 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee800b6c0 con 0x7f8ef406fa50 2024-05-08T04:13:15.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.233+0000 7f8edbfff640 1 -- 172.21.15.170:0/3063088796 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ee8019070 con 0x7f8ef406fa50 2024-05-08T04:13:15.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.233+0000 7f8ef9bab640 1 -- 172.21.15.170:0/3063088796 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef413b1a0 con 0x7f8ef406fa50 2024-05-08T04:13:15.238 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.233+0000 7f8edbfff640 1 -- 172.21.15.170:0/3063088796 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee800ba80 con 0x7f8ef406fa50 2024-05-08T04:13:15.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.233+0000 7f8ef9bab640 1 -- 172.21.15.170:0/3063088796 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef41dd950 con 0x7f8ef406fa50 2024-05-08T04:13:15.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.234+0000 7f8ef9bab640 1 -- 172.21.15.170:0/3063088796 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ef406e9e0 con 0x7f8ef406fa50 2024-05-08T04:13:15.239 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.235+0000 7f8edbfff640 1 -- 172.21.15.170:0/3063088796 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f8ee800bc20 con 0x7f8ef406fa50 2024-05-08T04:13:15.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.235+0000 7f8edbfff640 1 --2- 172.21.15.170:0/3063088796 >> v2:172.21.15.187:6816/101007897 conn(0x7f8ee0076290 0x7f8ee0078750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:15.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.236+0000 7f8edbfff640 1 -- 172.21.15.170:0/3063088796 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 4525+0+0 (secure 0 0 0) 0x7f8ee80974f0 con 0x7f8ef406fa50 2024-05-08T04:13:15.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.236+0000 7f8ef37fe640 1 --2- 172.21.15.170:0/3063088796 >> v2:172.21.15.187:6816/101007897 conn(0x7f8ee0076290 0x7f8ee0078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:15.240 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.236+0000 7f8ef37fe640 1 --2- 172.21.15.170:0/3063088796 >> v2:172.21.15.187:6816/101007897 conn(0x7f8ee0076290 0x7f8ee0078750 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4036000 tx=0x7f8ee4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:15.242 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.240+0000 7f8edbfff640 1 -- 172.21.15.170:0/3063088796 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8ee8061380 con 0x7f8ef406fa50 2024-05-08T04:13:15.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.417+0000 7f8ef9bab640 1 -- 172.21.15.170:0/3063088796 --> v2:172.21.15.187:6816/101007897 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8ef4145fb0 con 0x7f8ee0076290 2024-05-08T04:13:15.422 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.421+0000 7f8edbfff640 1 -- 172.21.15.170:0/3063088796 <== mgr.4107 v2:172.21.15.187:6816/101007897 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+117676 (secure 0 0 0) 0x7f8ef4145fb0 con 0x7f8ee0076290 2024-05-08T04:13:15.424 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:13:15.424 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-05-08T04:13:15.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.423+0000 7f8ef9bab640 1 -- 172.21.15.170:0/3063088796 >> v2:172.21.15.187:6816/101007897 conn(0x7f8ee0076290 msgr2=0x7f8ee0078750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:15.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.423+0000 7f8ef9bab640 1 --2- 172.21.15.170:0/3063088796 >> v2:172.21.15.187:6816/101007897 conn(0x7f8ee0076290 0x7f8ee0078750 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4036000 tx=0x7f8ee4030000 comp rx=0 tx=0).stop 2024-05-08T04:13:15.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.423+0000 7f8ef9bab640 1 -- 172.21.15.170:0/3063088796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ef406fa50 msgr2=0x7f8ef4132a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:15.426 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.423+0000 7f8ef9bab640 1 --2- 172.21.15.170:0/3063088796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ef406fa50 0x7f8ef4132a60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800d800 tx=0x7f8ee800dcc0 comp rx=0 tx=0).stop 2024-05-08T04:13:15.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.423+0000 7f8ef9bab640 1 -- 172.21.15.170:0/3063088796 shutdown_connections 2024-05-08T04:13:15.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.423+0000 7f8ef9bab640 1 --2- 172.21.15.170:0/3063088796 >> v2:172.21.15.187:6816/101007897 conn(0x7f8ee0076290 0x7f8ee0078750 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:15.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.423+0000 7f8ef9bab640 1 --2- 172.21.15.170:0/3063088796 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8ef4072dc0 0x7f8ef4132fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:15.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.423+0000 7f8ef9bab640 1 --2- 172.21.15.170:0/3063088796 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ef406fa50 0x7f8ef4132a60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:15.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.423+0000 7f8ef9bab640 1 --2- 172.21.15.170:0/3063088796 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8ef406f670 0x7f8ef41dcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:15.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.423+0000 7f8ef9bab640 1 -- 172.21.15.170:0/3063088796 >> 172.21.15.170:0/3063088796 conn(0x7f8ef40b4490 msgr2=0x7f8ef4072500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:15.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.423+0000 7f8ef9bab640 1 -- 172.21.15.170:0/3063088796 shutdown_connections 2024-05-08T04:13:15.427 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.423+0000 7f8ef9bab640 1 -- 172.21.15.170:0/3063088796 wait complete. 2024-05-08T04:13:15.437 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2024-05-08T04:13:14.241795+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3581805864,"num_objects":883,"num_object_clones":0,"num_object_copies":1766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":883,"num_whiteouts":0,"num_read":2570,"num_read_kb":2570,"num_write":164609,"num_write_kb":5657393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167289,"ondisk_log_size":167289,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9338120,"kb_used_data":6665288,"kb_used_omap":12,"kb_used_meta":2672755,"kb_avail":745636600,"statfs":{"total":773094113280,"available":763531878400,"internally_reserved":0,"allocated":6825254912,"data_stored":6862685808,"data_compressed":194295728,"data_compressed_allocated":202113024,"data_compressed_original":244916224,"omap_allocated":12751,"internal_metadata":2736901681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20480,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.525470"},"pg_stats":[{"pgid":"3.1f","version":"21'3951","reported_seq":3984,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.165552+0000","last_change":"2024-05-08T04:08:27.732101+0000","last_active":"2024-05-08T04:11:58.165552+0000","last_peered":"2024-05-08T04:11:58.165552+0000","last_clean":"2024-05-08T04:11:58.165552+0000","last_became_active":"2024-05-08T04:08:27.731781+0000","last_became_peered":"2024-05-08T04:08:27.731781+0000","last_unstale":"2024-05-08T04:11:58.165552+0000","last_undegraded":"2024-05-08T04:11:58.165552+0000","last_fullsized":"2024-05-08T04:11:58.165552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":3951,"log_dups_size":0,"ondisk_log_size":3951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:46:39.944802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":62,"num_read_kb":52,"num_write":3964,"num_write_kb":169350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"22'7380","reported_seq":7398,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:12:05.363320+0000","last_change":"2024-05-08T04:08:27.731984+0000","last_active":"2024-05-08T04:12:05.363320+0000","last_peered":"2024-05-08T04:12:05.363320+0000","last_clean":"2024-05-08T04:12:05.363320+0000","last_became_active":"2024-05-08T04:08:27.731598+0000","last_became_peered":"2024-05-08T04:08:27.731598+0000","last_unstale":"2024-05-08T04:12:05.363320+0000","last_undegraded":"2024-05-08T04:12:05.363320+0000","last_fullsized":"2024-05-08T04:12:05.363320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":7380,"log_dups_size":0,"ondisk_log_size":7380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:41:22.212562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":91594752,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":7365,"num_write_kb":213614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"22'5764","reported_seq":5793,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:12:10.049692+0000","last_change":"2024-05-08T04:08:27.732284+0000","last_active":"2024-05-08T04:12:10.049692+0000","last_peered":"2024-05-08T04:12:10.049692+0000","last_clean":"2024-05-08T04:12:10.049692+0000","last_became_active":"2024-05-08T04:08:27.732006+0000","last_became_peered":"2024-05-08T04:08:27.732006+0000","last_unstale":"2024-05-08T04:12:10.049692+0000","last_undegraded":"2024-05-08T04:12:10.049692+0000","last_fullsized":"2024-05-08T04:12:10.049692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":5764,"log_dups_size":0,"ondisk_log_size":5764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:24:55.281049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":56,"num_read_kb":44,"num_write":5770,"num_write_kb":177284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"21'4466","reported_seq":4488,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.166652+0000","last_change":"2024-05-08T04:08:27.733083+0000","last_active":"2024-05-08T04:11:58.166652+0000","last_peered":"2024-05-08T04:11:58.166652+0000","last_clean":"2024-05-08T04:11:58.166652+0000","last_became_active":"2024-05-08T04:08:27.732710+0000","last_became_peered":"2024-05-08T04:08:27.732710+0000","last_unstale":"2024-05-08T04:11:58.166652+0000","last_undegraded":"2024-05-08T04:11:58.166652+0000","last_fullsized":"2024-05-08T04:11:58.166652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":4466,"log_dups_size":0,"ondisk_log_size":4466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:40:43.219199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125829120,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":4481,"num_write_kb":186066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"22'4173","reported_seq":4190,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.397733+0000","last_change":"2024-05-08T04:08:27.726522+0000","last_active":"2024-05-08T04:11:58.397733+0000","last_peered":"2024-05-08T04:11:58.397733+0000","last_clean":"2024-05-08T04:11:58.397733+0000","last_became_active":"2024-05-08T04:08:27.726232+0000","last_became_peered":"2024-05-08T04:08:27.726232+0000","last_unstale":"2024-05-08T04:11:58.397733+0000","last_undegraded":"2024-05-08T04:11:58.397733+0000","last_fullsized":"2024-05-08T04:11:58.397733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":4173,"log_dups_size":0,"ondisk_log_size":4173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:07.708671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120203616,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":4136,"num_write_kb":200298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"22'6140","reported_seq":6158,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:12:03.565212+0000","last_change":"2024-05-08T04:08:27.722514+0000","last_active":"2024-05-08T04:12:03.565212+0000","last_peered":"2024-05-08T04:12:03.565212+0000","last_clean":"2024-05-08T04:12:03.565212+0000","last_became_active":"2024-05-08T04:08:27.721701+0000","last_became_peered":"2024-05-08T04:08:27.721701+0000","last_unstale":"2024-05-08T04:12:03.565212+0000","last_undegraded":"2024-05-08T04:12:03.565212+0000","last_fullsized":"2024-05-08T04:12:03.565212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":6140,"log_dups_size":0,"ondisk_log_size":6140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:44.554641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":6123,"num_write_kb":209934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"22'5185","reported_seq":5203,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:12:14.567740+0000","last_change":"2024-05-08T04:08:27.728824+0000","last_active":"2024-05-08T04:12:14.567740+0000","last_peered":"2024-05-08T04:12:14.567740+0000","last_clean":"2024-05-08T04:12:14.567740+0000","last_became_active":"2024-05-08T04:08:27.728564+0000","last_became_peered":"2024-05-08T04:08:27.728564+0000","last_unstale":"2024-05-08T04:12:14.567740+0000","last_undegraded":"2024-05-08T04:12:14.567740+0000","last_fullsized":"2024-05-08T04:12:14.567740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":5185,"log_dups_size":0,"ondisk_log_size":5185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:55.969304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142606336,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":5146,"num_write_kb":193669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"22'7560","reported_seq":7579,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:12:14.637052+0000","last_change":"2024-05-08T04:08:27.728565+0000","last_active":"2024-05-08T04:12:14.637052+0000","last_peered":"2024-05-08T04:12:14.637052+0000","last_clean":"2024-05-08T04:12:14.637052+0000","last_became_active":"2024-05-08T04:08:27.728205+0000","last_became_peered":"2024-05-08T04:08:27.728205+0000","last_unstale":"2024-05-08T04:12:14.637052+0000","last_undegraded":"2024-05-08T04:12:14.637052+0000","last_fullsized":"2024-05-08T04:12:14.637052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":7560,"log_dups_size":0,"ondisk_log_size":7560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:22:53.989163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142606336,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":12,"num_read_kb":135,"num_write":7569,"num_write_kb":191194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"22'5903","reported_seq":5918,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:12:17.354833+0000","last_change":"2024-05-08T04:08:27.732104+0000","last_active":"2024-05-08T04:12:17.354833+0000","last_peered":"2024-05-08T04:12:17.354833+0000","last_clean":"2024-05-08T04:12:17.354833+0000","last_became_active":"2024-05-08T04:08:27.731769+0000","last_became_peered":"2024-05-08T04:08:27.731769+0000","last_unstale":"2024-05-08T04:12:17.354833+0000","last_undegraded":"2024-05-08T04:12:17.354833+0000","last_fullsized":"2024-05-08T04:12:17.354833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":5903,"log_dups_size":0,"ondisk_log_size":5903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:21:26.194969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5896,"num_write_kb":160193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"22'4607","reported_seq":4623,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.338176+0000","last_change":"2024-05-08T04:08:27.731045+0000","last_active":"2024-05-08T04:11:58.338176+0000","last_peered":"2024-05-08T04:11:58.338176+0000","last_clean":"2024-05-08T04:11:58.338176+0000","last_became_active":"2024-05-08T04:08:27.730749+0000","last_became_peered":"2024-05-08T04:08:27.730749+0000","last_unstale":"2024-05-08T04:11:58.338176+0000","last_undegraded":"2024-05-08T04:11:58.338176+0000","last_fullsized":"2024-05-08T04:11:58.338176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":4607,"log_dups_size":0,"ondisk_log_size":4607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:17:07.247072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":127271424,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":4592,"num_write_kb":174343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"22'7467","reported_seq":7793,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:12:13.235858+0000","last_change":"2024-05-08T04:08:27.729609+0000","last_active":"2024-05-08T04:12:13.235858+0000","last_peered":"2024-05-08T04:12:13.235858+0000","last_clean":"2024-05-08T04:12:13.235858+0000","last_became_active":"2024-05-08T04:08:27.729318+0000","last_became_peered":"2024-05-08T04:08:27.729318+0000","last_unstale":"2024-05-08T04:12:13.235858+0000","last_undegraded":"2024-05-08T04:12:13.235858+0000","last_fullsized":"2024-05-08T04:12:13.235858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":7467,"log_dups_size":0,"ondisk_log_size":7467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:17:45.553221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112068432,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1045,"num_read_kb":996,"num_write":7685,"num_write_kb":175598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"22'6085","reported_seq":6101,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:12:07.645890+0000","last_change":"2024-05-08T04:08:27.730319+0000","last_active":"2024-05-08T04:12:07.645890+0000","last_peered":"2024-05-08T04:12:07.645890+0000","last_clean":"2024-05-08T04:12:07.645890+0000","last_became_active":"2024-05-08T04:08:27.729781+0000","last_became_peered":"2024-05-08T04:08:27.729781+0000","last_unstale":"2024-05-08T04:12:07.645890+0000","last_undegraded":"2024-05-08T04:12:07.645890+0000","last_fullsized":"2024-05-08T04:12:07.645890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":6085,"log_dups_size":0,"ondisk_log_size":6085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:29:26.256991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123143848,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6035,"num_write_kb":178933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.167611+0000","last_change":"2024-05-08T04:08:21.697423+0000","last_active":"2024-05-08T04:11:58.167611+0000","last_peered":"2024-05-08T04:11:58.167611+0000","last_clean":"2024-05-08T04:11:58.167611+0000","last_became_active":"2024-05-08T04:08:21.697149+0000","last_became_peered":"2024-05-08T04:08:21.697149+0000","last_unstale":"2024-05-08T04:11:58.167611+0000","last_undegraded":"2024-05-08T04:11:58.167611+0000","last_fullsized":"2024-05-08T04:11:58.167611+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:59:38.354372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"22'7339","reported_seq":7378,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:12:19.316202+0000","last_change":"2024-05-08T04:08:27.725473+0000","last_active":"2024-05-08T04:12:19.316202+0000","last_peered":"2024-05-08T04:12:19.316202+0000","last_clean":"2024-05-08T04:12:19.316202+0000","last_became_active":"2024-05-08T04:08:27.725165+0000","last_became_peered":"2024-05-08T04:08:27.725165+0000","last_unstale":"2024-05-08T04:12:19.316202+0000","last_undegraded":"2024-05-08T04:12:19.316202+0000","last_fullsized":"2024-05-08T04:12:19.316202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":7339,"log_dups_size":0,"ondisk_log_size":7339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:06:19.688474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142738448,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":74,"num_read_kb":59,"num_write":7371,"num_write_kb":187653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.137646+0000","last_change":"2024-05-08T04:08:21.696942+0000","last_active":"2024-05-08T04:11:58.137646+0000","last_peered":"2024-05-08T04:11:58.137646+0000","last_clean":"2024-05-08T04:11:58.137646+0000","last_became_active":"2024-05-08T04:08:21.696632+0000","last_became_peered":"2024-05-08T04:08:21.696632+0000","last_unstale":"2024-05-08T04:11:58.137646+0000","last_undegraded":"2024-05-08T04:11:58.137646+0000","last_fullsized":"2024-05-08T04:11:58.137646+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:23:25.156383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"22'3710","reported_seq":3726,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.394698+0000","last_change":"2024-05-08T04:08:27.730834+0000","last_active":"2024-05-08T04:11:58.394698+0000","last_peered":"2024-05-08T04:11:58.394698+0000","last_clean":"2024-05-08T04:11:58.394698+0000","last_became_active":"2024-05-08T04:08:27.730530+0000","last_became_peered":"2024-05-08T04:08:27.730530+0000","last_unstale":"2024-05-08T04:11:58.394698+0000","last_undegraded":"2024-05-08T04:11:58.394698+0000","last_fullsized":"2024-05-08T04:11:58.394698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":3710,"log_dups_size":0,"ondisk_log_size":3710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:51:02.242727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3661,"num_write_kb":168963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.138336+0000","last_change":"2024-05-08T04:08:21.696814+0000","last_active":"2024-05-08T04:11:58.138336+0000","last_peered":"2024-05-08T04:11:58.138336+0000","last_clean":"2024-05-08T04:11:58.138336+0000","last_became_active":"2024-05-08T04:08:21.696403+0000","last_became_peered":"2024-05-08T04:08:21.696403+0000","last_unstale":"2024-05-08T04:11:58.138336+0000","last_undegraded":"2024-05-08T04:11:58.138336+0000","last_fullsized":"2024-05-08T04:11:58.138336+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:02:04.075137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"22'6934","reported_seq":6950,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:12:16.551309+0000","last_change":"2024-05-08T04:08:27.721621+0000","last_active":"2024-05-08T04:12:16.551309+0000","last_peered":"2024-05-08T04:12:16.551309+0000","last_clean":"2024-05-08T04:12:16.551309+0000","last_became_active":"2024-05-08T04:08:27.721303+0000","last_became_peered":"2024-05-08T04:08:27.721303+0000","last_unstale":"2024-05-08T04:12:16.551309+0000","last_undegraded":"2024-05-08T04:12:16.551309+0000","last_fullsized":"2024-05-08T04:12:16.551309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":6934,"log_dups_size":0,"ondisk_log_size":6934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:59:37.780694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6930,"num_write_kb":193544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.137534+0000","last_change":"2024-05-08T04:08:21.693307+0000","last_active":"2024-05-08T04:11:58.137534+0000","last_peered":"2024-05-08T04:11:58.137534+0000","last_clean":"2024-05-08T04:11:58.137534+0000","last_became_active":"2024-05-08T04:08:21.693014+0000","last_became_peered":"2024-05-08T04:08:21.693014+0000","last_unstale":"2024-05-08T04:11:58.137534+0000","last_undegraded":"2024-05-08T04:11:58.137534+0000","last_fullsized":"2024-05-08T04:11:58.137534+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:56.218680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"22'4514","reported_seq":4531,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:12:18.455836+0000","last_change":"2024-05-08T04:08:27.729798+0000","last_active":"2024-05-08T04:12:18.455836+0000","last_peered":"2024-05-08T04:12:18.455836+0000","last_clean":"2024-05-08T04:12:18.455836+0000","last_became_active":"2024-05-08T04:08:27.729504+0000","last_became_peered":"2024-05-08T04:08:27.729504+0000","last_unstale":"2024-05-08T04:12:18.455836+0000","last_undegraded":"2024-05-08T04:12:18.455836+0000","last_fullsized":"2024-05-08T04:12:18.455836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":4514,"log_dups_size":0,"ondisk_log_size":4514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:58:47.867425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96535048,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":4515,"num_write_kb":160844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"21'6272","reported_seq":6401,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.140568+0000","last_change":"2024-05-08T04:08:27.726831+0000","last_active":"2024-05-08T04:11:58.140568+0000","last_peered":"2024-05-08T04:11:58.140568+0000","last_clean":"2024-05-08T04:11:58.140568+0000","last_became_active":"2024-05-08T04:08:27.726527+0000","last_became_peered":"2024-05-08T04:08:27.726527+0000","last_unstale":"2024-05-08T04:11:58.140568+0000","last_undegraded":"2024-05-08T04:11:58.140568+0000","last_fullsized":"2024-05-08T04:11:58.140568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":6272,"log_dups_size":0,"ondisk_log_size":6272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:26:39.243499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":126223376,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":153,"num_read_kb":139,"num_write":6308,"num_write_kb":247140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.166594+0000","last_change":"2024-05-08T04:08:21.699221+0000","last_active":"2024-05-08T04:11:58.166594+0000","last_peered":"2024-05-08T04:11:58.166594+0000","last_clean":"2024-05-08T04:11:58.166594+0000","last_became_active":"2024-05-08T04:08:21.699013+0000","last_became_peered":"2024-05-08T04:08:21.699013+0000","last_unstale":"2024-05-08T04:11:58.166594+0000","last_undegraded":"2024-05-08T04:11:58.166594+0000","last_fullsized":"2024-05-08T04:11:58.166594+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:23:08.040183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"22'3067","reported_seq":3083,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.391790+0000","last_change":"2024-05-08T04:08:27.723409+0000","last_active":"2024-05-08T04:11:58.391790+0000","last_peered":"2024-05-08T04:11:58.391790+0000","last_clean":"2024-05-08T04:11:58.391790+0000","last_became_active":"2024-05-08T04:08:27.723118+0000","last_became_peered":"2024-05-08T04:08:27.723118+0000","last_unstale":"2024-05-08T04:11:58.391790+0000","last_undegraded":"2024-05-08T04:11:58.391790+0000","last_fullsized":"2024-05-08T04:11:58.391790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":3067,"log_dups_size":0,"ondisk_log_size":3067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:31:49.593896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3019,"num_write_kb":152195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.137838+0000","last_change":"2024-05-08T04:08:21.799283+0000","last_active":"2024-05-08T04:11:58.137838+0000","last_peered":"2024-05-08T04:11:58.137838+0000","last_clean":"2024-05-08T04:11:58.137838+0000","last_became_active":"2024-05-08T04:08:21.798995+0000","last_became_peered":"2024-05-08T04:08:21.798995+0000","last_unstale":"2024-05-08T04:11:58.137838+0000","last_undegraded":"2024-05-08T04:11:58.137838+0000","last_fullsized":"2024-05-08T04:11:58.137838+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:37:16.795888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"22'3228","reported_seq":3245,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.397567+0000","last_change":"2024-05-08T04:08:27.726742+0000","last_active":"2024-05-08T04:11:58.397567+0000","last_peered":"2024-05-08T04:11:58.397567+0000","last_clean":"2024-05-08T04:11:58.397567+0000","last_became_active":"2024-05-08T04:08:27.726457+0000","last_became_peered":"2024-05-08T04:08:27.726457+0000","last_unstale":"2024-05-08T04:11:58.397567+0000","last_undegraded":"2024-05-08T04:11:58.397567+0000","last_fullsized":"2024-05-08T04:11:58.397567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":3228,"log_dups_size":0,"ondisk_log_size":3228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:36:59.353164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104923656,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":3211,"num_write_kb":154644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.137996+0000","last_change":"2024-05-08T04:08:21.697070+0000","last_active":"2024-05-08T04:11:58.137996+0000","last_peered":"2024-05-08T04:11:58.137996+0000","last_clean":"2024-05-08T04:11:58.137996+0000","last_became_active":"2024-05-08T04:08:21.696370+0000","last_became_peered":"2024-05-08T04:08:21.696370+0000","last_unstale":"2024-05-08T04:11:58.137996+0000","last_undegraded":"2024-05-08T04:11:58.137996+0000","last_fullsized":"2024-05-08T04:11:58.137996+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:54.981822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"22'5074","reported_seq":5090,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:12:11.590538+0000","last_change":"2024-05-08T04:08:27.732812+0000","last_active":"2024-05-08T04:12:11.590538+0000","last_peered":"2024-05-08T04:12:11.590538+0000","last_clean":"2024-05-08T04:12:11.590538+0000","last_became_active":"2024-05-08T04:08:27.732446+0000","last_became_peered":"2024-05-08T04:08:27.732446+0000","last_unstale":"2024-05-08T04:12:11.590538+0000","last_undegraded":"2024-05-08T04:12:11.590538+0000","last_fullsized":"2024-05-08T04:12:11.590538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":5074,"log_dups_size":0,"ondisk_log_size":5074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:34:38.276980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5070,"num_write_kb":160386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.166095+0000","last_change":"2024-05-08T04:08:21.696942+0000","last_active":"2024-05-08T04:11:58.166095+0000","last_peered":"2024-05-08T04:11:58.166095+0000","last_clean":"2024-05-08T04:11:58.166095+0000","last_became_active":"2024-05-08T04:08:21.696659+0000","last_became_peered":"2024-05-08T04:08:21.696659+0000","last_unstale":"2024-05-08T04:11:58.166095+0000","last_undegraded":"2024-05-08T04:11:58.166095+0000","last_fullsized":"2024-05-08T04:11:58.166095+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:50:44.209520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":114,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.137889+0000","last_change":"2024-05-08T04:08:20.691470+0000","last_active":"2024-05-08T04:11:58.137889+0000","last_peered":"2024-05-08T04:11:58.137889+0000","last_clean":"2024-05-08T04:11:58.137889+0000","last_became_active":"2024-05-08T04:08:20.691204+0000","last_became_peered":"2024-05-08T04:08:20.691204+0000","last_unstale":"2024-05-08T04:11:58.137889+0000","last_undegraded":"2024-05-08T04:11:58.137889+0000","last_fullsized":"2024-05-08T04:11:58.137889+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:19.679313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:19.679313+0000","last_clean_scrub_stamp":"2024-05-08T04:08:19.679313+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:52:31.185659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"22'3937","reported_seq":3958,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.392812+0000","last_change":"2024-05-08T04:08:27.729757+0000","last_active":"2024-05-08T04:11:58.392812+0000","last_peered":"2024-05-08T04:11:58.392812+0000","last_clean":"2024-05-08T04:11:58.392812+0000","last_became_active":"2024-05-08T04:08:27.729321+0000","last_became_peered":"2024-05-08T04:08:27.729321+0000","last_unstale":"2024-05-08T04:11:58.392812+0000","last_undegraded":"2024-05-08T04:11:58.392812+0000","last_fullsized":"2024-05-08T04:11:58.392812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":3937,"log_dups_size":0,"ondisk_log_size":3937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:12:28.413005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109182976,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":3879,"num_write_kb":165315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"22'9381","reported_seq":9518,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:12:49.841474+0000","last_change":"2024-05-08T04:08:27.721874+0000","last_active":"2024-05-08T04:12:49.841474+0000","last_peered":"2024-05-08T04:12:49.841474+0000","last_clean":"2024-05-08T04:12:49.841474+0000","last_became_active":"2024-05-08T04:08:27.721534+0000","last_became_peered":"2024-05-08T04:08:27.721534+0000","last_unstale":"2024-05-08T04:12:49.841474+0000","last_undegraded":"2024-05-08T04:12:49.841474+0000","last_fullsized":"2024-05-08T04:12:49.841474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":9381,"log_dups_size":0,"ondisk_log_size":9381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:59:46.563205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":139724624,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":382,"num_read_kb":338,"num_write":6452,"num_write_kb":194667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"22'3827","reported_seq":3865,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:12:16.074345+0000","last_change":"2024-05-08T04:08:27.726164+0000","last_active":"2024-05-08T04:12:16.074345+0000","last_peered":"2024-05-08T04:12:16.074345+0000","last_clean":"2024-05-08T04:12:16.074345+0000","last_became_active":"2024-05-08T04:08:27.725833+0000","last_became_peered":"2024-05-08T04:08:27.725833+0000","last_unstale":"2024-05-08T04:12:16.074345+0000","last_undegraded":"2024-05-08T04:12:16.074345+0000","last_fullsized":"2024-05-08T04:12:16.074345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":3827,"log_dups_size":0,"ondisk_log_size":3827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:35.399549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":85458944,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":91,"num_read_kb":137,"num_write":3837,"num_write_kb":155137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"21'4041","reported_seq":4081,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.165639+0000","last_change":"2024-05-08T04:08:27.727947+0000","last_active":"2024-05-08T04:11:58.165639+0000","last_peered":"2024-05-08T04:11:58.165639+0000","last_clean":"2024-05-08T04:11:58.165639+0000","last_became_active":"2024-05-08T04:08:27.727653+0000","last_became_peered":"2024-05-08T04:08:27.727653+0000","last_unstale":"2024-05-08T04:11:58.165639+0000","last_undegraded":"2024-05-08T04:11:58.165639+0000","last_fullsized":"2024-05-08T04:11:58.165639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":4041,"log_dups_size":0,"ondisk_log_size":4041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:14:37.705194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130023424,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":90,"num_read_kb":141,"num_write":4071,"num_write_kb":155757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"22'5322","reported_seq":5346,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.392917+0000","last_change":"2024-05-08T04:08:27.730956+0000","last_active":"2024-05-08T04:11:58.392917+0000","last_peered":"2024-05-08T04:11:58.392917+0000","last_clean":"2024-05-08T04:11:58.392917+0000","last_became_active":"2024-05-08T04:08:27.730569+0000","last_became_peered":"2024-05-08T04:08:27.730569+0000","last_unstale":"2024-05-08T04:11:58.392917+0000","last_undegraded":"2024-05-08T04:11:58.392917+0000","last_fullsized":"2024-05-08T04:11:58.392917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":5322,"log_dups_size":0,"ondisk_log_size":5322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:05:40.969984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":5289,"num_write_kb":148492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"22'4023","reported_seq":4053,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.390778+0000","last_change":"2024-05-08T04:08:27.733357+0000","last_active":"2024-05-08T04:11:58.390778+0000","last_peered":"2024-05-08T04:11:58.390778+0000","last_clean":"2024-05-08T04:11:58.390778+0000","last_became_active":"2024-05-08T04:08:27.733070+0000","last_became_peered":"2024-05-08T04:08:27.733070+0000","last_unstale":"2024-05-08T04:11:58.390778+0000","last_undegraded":"2024-05-08T04:11:58.390778+0000","last_fullsized":"2024-05-08T04:11:58.390778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":4023,"log_dups_size":0,"ondisk_log_size":4023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:33:08.617696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":103481344,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":53,"num_read_kb":105,"num_write":4128,"num_write_kb":186691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"22'3219","reported_seq":3313,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.434132+0000","last_change":"2024-05-08T04:08:27.730671+0000","last_active":"2024-05-08T04:11:58.434132+0000","last_peered":"2024-05-08T04:11:58.434132+0000","last_clean":"2024-05-08T04:11:58.434132+0000","last_became_active":"2024-05-08T04:08:27.730372+0000","last_became_peered":"2024-05-08T04:08:27.730372+0000","last_unstale":"2024-05-08T04:11:58.434132+0000","last_undegraded":"2024-05-08T04:11:58.434132+0000","last_fullsized":"2024-05-08T04:11:58.434132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":3219,"log_dups_size":0,"ondisk_log_size":3219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:55:59.085956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":78389776,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":301,"num_read_kb":226,"num_write":3489,"num_write_kb":113772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"22'4347","reported_seq":4366,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:13:03.293177+0000","last_change":"2024-05-08T04:08:27.729948+0000","last_active":"2024-05-08T04:13:03.293177+0000","last_peered":"2024-05-08T04:13:03.293177+0000","last_clean":"2024-05-08T04:13:03.293177+0000","last_became_active":"2024-05-08T04:08:27.729584+0000","last_became_peered":"2024-05-08T04:08:27.729584+0000","last_unstale":"2024-05-08T04:13:03.293177+0000","last_undegraded":"2024-05-08T04:13:03.293177+0000","last_fullsized":"2024-05-08T04:13:03.293177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":4347,"log_dups_size":0,"ondisk_log_size":4347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:35:18.017771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95482384,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":4309,"num_write_kb":147779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"22'5304","reported_seq":5328,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.392364+0000","last_change":"2024-05-08T04:08:27.727021+0000","last_active":"2024-05-08T04:11:58.392364+0000","last_peered":"2024-05-08T04:11:58.392364+0000","last_clean":"2024-05-08T04:11:58.392364+0000","last_became_active":"2024-05-08T04:08:27.726692+0000","last_became_peered":"2024-05-08T04:08:27.726692+0000","last_unstale":"2024-05-08T04:11:58.392364+0000","last_undegraded":"2024-05-08T04:11:58.392364+0000","last_fullsized":"2024-05-08T04:11:58.392364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":5304,"log_dups_size":0,"ondisk_log_size":5304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:16:39.270048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120979456,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":5269,"num_write_kb":245378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"21'2450","reported_seq":2466,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.138003+0000","last_change":"2024-05-08T04:08:27.727459+0000","last_active":"2024-05-08T04:11:58.138003+0000","last_peered":"2024-05-08T04:11:58.138003+0000","last_clean":"2024-05-08T04:11:58.138003+0000","last_became_active":"2024-05-08T04:08:27.727138+0000","last_became_peered":"2024-05-08T04:08:27.727138+0000","last_unstale":"2024-05-08T04:11:58.138003+0000","last_undegraded":"2024-05-08T04:11:58.138003+0000","last_fullsized":"2024-05-08T04:11:58.138003+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":2450,"log_dups_size":0,"ondisk_log_size":2450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:32:43.624016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2442,"num_write_kb":115332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"22'3788","reported_seq":3812,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.400216+0000","last_change":"2024-05-08T04:08:27.729355+0000","last_active":"2024-05-08T04:11:58.400216+0000","last_peered":"2024-05-08T04:11:58.400216+0000","last_clean":"2024-05-08T04:11:58.400216+0000","last_became_active":"2024-05-08T04:08:27.729024+0000","last_became_peered":"2024-05-08T04:08:27.729024+0000","last_unstale":"2024-05-08T04:11:58.400216+0000","last_undegraded":"2024-05-08T04:11:58.400216+0000","last_fullsized":"2024-05-08T04:11:58.400216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":3788,"log_dups_size":0,"ondisk_log_size":3788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:28:16.629952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96600064,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":3752,"num_write_kb":156932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"22'8759","reported_seq":8777,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:12:10.448806+0000","last_change":"2024-05-08T04:08:27.729279+0000","last_active":"2024-05-08T04:12:10.448806+0000","last_peered":"2024-05-08T04:12:10.448806+0000","last_clean":"2024-05-08T04:12:10.448806+0000","last_became_active":"2024-05-08T04:08:27.728925+0000","last_became_peered":"2024-05-08T04:08:27.728925+0000","last_unstale":"2024-05-08T04:12:10.448806+0000","last_undegraded":"2024-05-08T04:12:10.448806+0000","last_fullsized":"2024-05-08T04:12:10.448806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":8759,"log_dups_size":0,"ondisk_log_size":8759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:40:38.725967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163709456,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":8744,"num_write_kb":214668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3581215496,"num_objects":881,"num_object_clones":0,"num_object_copies":1762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":881,"num_whiteouts":0,"num_read":2504,"num_read_kb":2515,"num_write":164508,"num_write_kb":5655765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6823510016,"data_stored":6861080728,"data_compressed":194285506,"data_compressed_allocated":202047488,"data_compressed_original":243892224,"omap_allocated":0,"internal_metadata":0},"log_size":167217,"ondisk_log_size":167217,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":10222,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542205,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1406956,"kb_used_data":1010724,"kb_used_omap":1,"kb_used_meta":396222,"kb_avail":92964884,"statfs":{"total":96636764160,"available":95196041216,"internally_reserved":0,"allocated":1034981376,"data_stored":1042910983,"data_compressed":31421929,"data_compressed_allocated":32739328,"data_compressed_original":41480192,"omap_allocated":1589,"internal_metadata":405731787},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542205,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406896,"kb_used_data":259176,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":93964944,"statfs":{"total":96636764160,"available":96220102656,"internally_reserved":0,"allocated":265396224,"data_stored":266322235,"data_compressed":7220440,"data_compressed_allocated":7479296,"data_compressed_original":8728576,"omap_allocated":1594,"internal_metadata":151255494},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542205,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1269668,"kb_used_data":891484,"kb_used_omap":1,"kb_used_meta":378174,"kb_avail":93102172,"statfs":{"total":96636764160,"available":95336624128,"internally_reserved":0,"allocated":912879616,"data_stored":917907315,"data_compressed":26020264,"data_compressed_allocated":26992640,"data_compressed_original":32751616,"omap_allocated":1595,"internal_metadata":387250629},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542205,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1904424,"kb_used_data":1372192,"kb_used_omap":1,"kb_used_meta":532222,"kb_avail":92467416,"statfs":{"total":96636764160,"available":94686633984,"internally_reserved":0,"allocated":1405124608,"data_stored":1412898687,"data_compressed":41362870,"data_compressed_allocated":43028480,"data_compressed_original":51744768,"omap_allocated":1595,"internal_metadata":544995781},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542205,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1508028,"kb_used_data":1053492,"kb_used_omap":1,"kb_used_meta":454526,"kb_avail":92863812,"statfs":{"total":96636764160,"available":95092543488,"internally_reserved":0,"allocated":1078775808,"data_stored":1080567039,"data_compressed":21885706,"data_compressed_allocated":22720512,"data_compressed_original":25346048,"omap_allocated":1593,"internal_metadata":465435079},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542205,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":952204,"kb_used_data":673988,"kb_used_omap":1,"kb_used_meta":278206,"kb_avail":93419636,"statfs":{"total":96636764160,"available":95661707264,"internally_reserved":0,"allocated":690163712,"data_stored":694964323,"data_compressed":20980979,"data_compressed_allocated":21929984,"data_compressed_original":27267072,"omap_allocated":1595,"internal_metadata":284883397},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542205,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1461540,"kb_used_data":1079196,"kb_used_omap":1,"kb_used_meta":382334,"kb_avail":92910300,"statfs":{"total":96636764160,"available":95140147200,"internally_reserved":0,"allocated":1105096704,"data_stored":1112874703,"data_compressed":36587350,"data_compressed_allocated":38072320,"data_compressed_original":46706688,"omap_allocated":1595,"internal_metadata":391510469},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574910,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":428404,"kb_used_data":325036,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":93943436,"statfs":{"total":96636764160,"available":96198078464,"internally_reserved":0,"allocated":332836864,"data_stored":334240523,"data_compressed":8816190,"data_compressed_allocated":9150464,"data_compressed_original":10891264,"omap_allocated":1595,"internal_metadata":105839045},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5111,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5111,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1104838656,"data_stored":1112277372,"data_compressed":36582239,"data_compressed_allocated":38039552,"data_compressed_original":46194688,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":689958912,"data_stored":694895920,"data_compressed":20980979,"data_compressed_allocated":21929984,"data_compressed_original":27267072,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":332632064,"data_stored":334172120,"data_compressed":8816190,"data_compressed_allocated":9150464,"data_compressed_original":10891264,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1078571008,"data_stored":1080498636,"data_compressed":21885706,"data_compressed_allocated":22720512,"data_compressed_original":25346048,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1404919808,"data_stored":1412830284,"data_compressed":41362870,"data_compressed_allocated":43028480,"data_compressed_original":51744768,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":912674816,"data_stored":917838912,"data_compressed":26020264,"data_compressed_allocated":26992640,"data_compressed_original":32751616,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":265191424,"data_stored":266253832,"data_compressed":7220440,"data_compressed_allocated":7479296,"data_compressed_original":8728576,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1034723328,"data_stored":1042313652,"data_compressed":31416818,"data_compressed_allocated":32706560,"data_compressed_original":40968192,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:13:15.439 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T04:13:15.439 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:13:15.537 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.536+0000 7fbb36240640 1 Processor -- start 2024-05-08T04:13:15.538 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.536+0000 7fbb36240640 1 -- start start 2024-05-08T04:13:15.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.536+0000 7fbb36240640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbb3006ee70 0x7fbb3006f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:15.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.536+0000 7fbb36240640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fbb300739e0 0x7fbb30144820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:15.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.536+0000 7fbb36240640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb3006fc90 con 0x7fbb300735a0 2024-05-08T04:13:15.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.536+0000 7fbb36240640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7fbb30144d60 con 0x7fbb300739e0 2024-05-08T04:13:15.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.536+0000 7fbb36240640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fbb30072dc0 con 0x7fbb3006ee70 2024-05-08T04:13:15.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.537+0000 7fbb2ffff640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fbb300739e0 0x7fbb30144820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:15.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.537+0000 7fbb2effd640 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fbb300735a0 0x7fbb30144110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43718/0 (socket says 172.21.15.170:43718) 2024-05-08T04:13:15.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.537+0000 7fbb2effd640 1 -- 172.21.15.170:0/2715775288 learned_addr learned my addr 172.21.15.170:0/2715775288 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-05-08T04:13:15.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.537+0000 7fbb2ffff640 1 --2- 172.21.15.170:0/2715775288 >> v2:172.21.15.170:3300/0 conn(0x7fbb300739e0 0x7fbb30144820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49314/0 (socket says 172.21.15.170:49314) 2024-05-08T04:13:15.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.537+0000 7fbb2f7fe640 1 --2- 172.21.15.170:0/2715775288 >> v2:172.21.15.187:3300/0 conn(0x7fbb3006ee70 0x7fbb3006f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:15.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.537+0000 7fbb2ffff640 1 -- 172.21.15.170:0/2715775288 >> v2:172.21.15.187:3300/0 conn(0x7fbb3006ee70 msgr2=0x7fbb3006f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:13:15.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.537+0000 7fbb2ffff640 1 --2- 172.21.15.170:0/2715775288 >> v2:172.21.15.187:3300/0 conn(0x7fbb3006ee70 0x7fbb3006f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:15.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.537+0000 7fbb2ffff640 1 -- 172.21.15.170:0/2715775288 >> v1:172.21.15.170:6789/0 conn(0x7fbb300735a0 legacy=0x7fbb30144110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:15.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.537+0000 7fbb2ffff640 1 -- 172.21.15.170:0/2715775288 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb300b5100 con 0x7fbb300739e0 2024-05-08T04:13:15.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.537+0000 7fbb2e7fc640 1 -- 172.21.15.170:0/2715775288 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2623155608 0 0) 0x7fbb3006fc90 con 0x7fbb300735a0 2024-05-08T04:13:15.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.538+0000 7fbb2ffff640 1 --2- 172.21.15.170:0/2715775288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb300739e0 0x7fbb30144820 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbb3006d4f0 tx=0x7fbb2802bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf28bb8db9c30462 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:15.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.538+0000 7fbb2e7fc640 1 -- 172.21.15.170:0/2715775288 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb28039070 con 0x7fbb300739e0 2024-05-08T04:13:15.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.538+0000 7fbb2e7fc640 1 -- 172.21.15.170:0/2715775288 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb2802cb60 con 0x7fbb300739e0 2024-05-08T04:13:15.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.538+0000 7fbb2e7fc640 1 -- 172.21.15.170:0/2715775288 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb28034440 con 0x7fbb300739e0 2024-05-08T04:13:15.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.538+0000 7fbb36240640 1 -- 172.21.15.170:0/2715775288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb300739e0 msgr2=0x7fbb30144820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:15.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.538+0000 7fbb36240640 1 --2- 172.21.15.170:0/2715775288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb300739e0 0x7fbb30144820 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbb3006d4f0 tx=0x7fbb2802bdb0 comp rx=0 tx=0).stop 2024-05-08T04:13:15.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.538+0000 7fbb36240640 1 -- 172.21.15.170:0/2715775288 shutdown_connections 2024-05-08T04:13:15.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.538+0000 7fbb36240640 1 --2- 172.21.15.170:0/2715775288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb300739e0 0x7fbb30144820 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:15.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.538+0000 7fbb36240640 1 --2- 172.21.15.170:0/2715775288 >> v2:172.21.15.187:3300/0 conn(0x7fbb3006ee70 0x7fbb3006f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:15.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.538+0000 7fbb36240640 1 -- 172.21.15.170:0/2715775288 >> 172.21.15.170:0/2715775288 conn(0x7fbb300b45e0 msgr2=0x7fbb300b49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:15.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.538+0000 7fbb36240640 1 -- 172.21.15.170:0/2715775288 shutdown_connections 2024-05-08T04:13:15.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.538+0000 7fbb36240640 1 -- 172.21.15.170:0/2715775288 wait complete. 2024-05-08T04:13:15.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.539+0000 7fbb36240640 1 Processor -- start 2024-05-08T04:13:15.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.540+0000 7fbb36240640 1 -- start start 2024-05-08T04:13:15.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.540+0000 7fbb36240640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbb3006ee70 0x7fbb301411a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:15.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.540+0000 7fbb36240640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb300735a0 0x7fbb3013d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:15.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.540+0000 7fbb36240640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb300739e0 0x7fbb3013eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:15.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.540+0000 7fbb36240640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3006f790 con 0x7fbb300739e0 2024-05-08T04:13:15.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.540+0000 7fbb36240640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbb3006f910 con 0x7fbb300735a0 2024-05-08T04:13:15.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.540+0000 7fbb36240640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fbb3014e780 con 0x7fbb3006ee70 2024-05-08T04:13:15.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.540+0000 7fbb2f7fe640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbb3006ee70 0x7fbb301411a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:15.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.540+0000 7fbb2ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb300739e0 0x7fbb3013eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:15.544 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.540+0000 7fbb2ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb300739e0 0x7fbb3013eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49320/0 (socket says 172.21.15.170:49320) 2024-05-08T04:13:15.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.540+0000 7fbb2f7fe640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbb3006ee70 0x7fbb301411a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:47950/0 (socket says 172.21.15.170:47950) 2024-05-08T04:13:15.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.540+0000 7fbb2ffff640 1 -- 172.21.15.170:0/768499231 learned_addr learned my addr 172.21.15.170:0/768499231 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:13:15.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.540+0000 7fbb2ffff640 1 -- 172.21.15.170:0/768499231 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbb3006ee70 msgr2=0x7fbb301411a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:15.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.540+0000 7fbb2ffff640 1 --2- 172.21.15.170:0/768499231 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbb3006ee70 0x7fbb301411a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:15.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.540+0000 7fbb2ffff640 1 -- 172.21.15.170:0/768499231 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb300735a0 msgr2=0x7fbb3013d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:15.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.540+0000 7fbb2ffff640 1 --2- 172.21.15.170:0/768499231 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb300735a0 0x7fbb3013d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:15.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.540+0000 7fbb2ffff640 1 -- 172.21.15.170:0/768499231 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb301404c0 con 0x7fbb300739e0 2024-05-08T04:13:15.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.541+0000 7fbb2ffff640 1 --2- 172.21.15.170:0/768499231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb300739e0 0x7fbb3013eb20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbb28009e80 tx=0x7fbb28004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:15.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.541+0000 7fbb17fff640 1 -- 172.21.15.170:0/768499231 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb28044020 con 0x7fbb300739e0 2024-05-08T04:13:15.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.541+0000 7fbb17fff640 1 -- 172.21.15.170:0/768499231 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb2803a040 con 0x7fbb300739e0 2024-05-08T04:13:15.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.541+0000 7fbb36240640 1 -- 172.21.15.170:0/768499231 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb3013e600 con 0x7fbb300739e0 2024-05-08T04:13:15.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.541+0000 7fbb17fff640 1 -- 172.21.15.170:0/768499231 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb28039050 con 0x7fbb300739e0 2024-05-08T04:13:15.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.541+0000 7fbb36240640 1 -- 172.21.15.170:0/768499231 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb301d9900 con 0x7fbb300739e0 2024-05-08T04:13:15.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.542+0000 7fbb36240640 1 -- 172.21.15.170:0/768499231 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb30068f30 con 0x7fbb300739e0 2024-05-08T04:13:15.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.542+0000 7fbb17fff640 1 -- 172.21.15.170:0/768499231 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7fbb2802cd10 con 0x7fbb300739e0 2024-05-08T04:13:15.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.543+0000 7fbb17fff640 1 --2- 172.21.15.170:0/768499231 >> v2:172.21.15.187:6816/101007897 conn(0x7fbb1c076290 0x7fbb1c078750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:15.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.543+0000 7fbb17fff640 1 -- 172.21.15.170:0/768499231 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 4525+0+0 (secure 0 0 0) 0x7fbb280b9400 con 0x7fbb300739e0 2024-05-08T04:13:15.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.543+0000 7fbb2f7fe640 1 --2- 172.21.15.170:0/768499231 >> v2:172.21.15.187:6816/101007897 conn(0x7fbb1c076290 0x7fbb1c078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:15.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.544+0000 7fbb2f7fe640 1 --2- 172.21.15.170:0/768499231 >> v2:172.21.15.187:6816/101007897 conn(0x7fbb1c076290 0x7fbb1c078750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009aa0 tx=0x7fbb20009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:15.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.547+0000 7fbb17fff640 1 -- 172.21.15.170:0/768499231 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbb30068f30 con 0x7fbb300739e0 2024-05-08T04:13:15.729 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.727+0000 7fbb36240640 1 -- 172.21.15.170:0/768499231 --> v2:172.21.15.187:6816/101007897 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbb3006e9e0 con 0x7fbb1c076290 2024-05-08T04:13:15.732 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.731+0000 7fbb17fff640 1 -- 172.21.15.170:0/768499231 <== mgr.4107 v2:172.21.15.187:6816/101007897 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+117781 (secure 0 0 0) 0x7fbb3006e9e0 con 0x7fbb1c076290 2024-05-08T04:13:15.733 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:13:15.734 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-05-08T04:13:15.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.733+0000 7fbb36240640 1 -- 172.21.15.170:0/768499231 >> v2:172.21.15.187:6816/101007897 conn(0x7fbb1c076290 msgr2=0x7fbb1c078750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:15.735 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.733+0000 7fbb36240640 1 --2- 172.21.15.170:0/768499231 >> v2:172.21.15.187:6816/101007897 conn(0x7fbb1c076290 0x7fbb1c078750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009aa0 tx=0x7fbb20009190 comp rx=0 tx=0).stop 2024-05-08T04:13:15.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.733+0000 7fbb36240640 1 -- 172.21.15.170:0/768499231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb300739e0 msgr2=0x7fbb3013eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:15.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.733+0000 7fbb36240640 1 --2- 172.21.15.170:0/768499231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb300739e0 0x7fbb3013eb20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbb28009e80 tx=0x7fbb28004a40 comp rx=0 tx=0).stop 2024-05-08T04:13:15.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.733+0000 7fbb36240640 1 -- 172.21.15.170:0/768499231 shutdown_connections 2024-05-08T04:13:15.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.733+0000 7fbb36240640 1 --2- 172.21.15.170:0/768499231 >> v2:172.21.15.187:6816/101007897 conn(0x7fbb1c076290 0x7fbb1c078750 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:15.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.733+0000 7fbb36240640 1 --2- 172.21.15.170:0/768499231 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbb300735a0 0x7fbb3013d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:15.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.733+0000 7fbb36240640 1 --2- 172.21.15.170:0/768499231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb300739e0 0x7fbb3013eb20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:15.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.733+0000 7fbb36240640 1 --2- 172.21.15.170:0/768499231 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbb3006ee70 0x7fbb301411a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:15.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.733+0000 7fbb36240640 1 -- 172.21.15.170:0/768499231 >> 172.21.15.170:0/768499231 conn(0x7fbb300b45e0 msgr2=0x7fbb30135b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:15.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.733+0000 7fbb36240640 1 -- 172.21.15.170:0/768499231 shutdown_connections 2024-05-08T04:13:15.736 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.733+0000 7fbb36240640 1 -- 172.21.15.170:0/768499231 wait complete. 2024-05-08T04:13:15.747 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2024-05-08T04:13:15.719361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3581797144,"num_objects":882,"num_object_clones":0,"num_object_copies":1764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":882,"num_whiteouts":0,"num_read":2636,"num_read_kb":2641,"num_write":164626,"num_write_kb":5657401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167333,"ondisk_log_size":167333,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9338188,"kb_used_data":6665336,"kb_used_omap":12,"kb_used_meta":2672755,"kb_avail":745636532,"statfs":{"total":773094113280,"available":763531808768,"internally_reserved":0,"allocated":6825304064,"data_stored":6862700928,"data_compressed":194295728,"data_compressed_allocated":202113024,"data_compressed_original":244916224,"omap_allocated":12751,"internal_metadata":2736901681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8720,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":66,"num_read_kb":71,"num_write":17,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002620"},"pg_stats":[{"pgid":"3.1f","version":"21'3951","reported_seq":3986,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.244099+0000","last_change":"2024-05-08T04:08:27.732101+0000","last_active":"2024-05-08T04:13:14.244099+0000","last_peered":"2024-05-08T04:13:14.244099+0000","last_clean":"2024-05-08T04:13:14.244099+0000","last_became_active":"2024-05-08T04:08:27.731781+0000","last_became_peered":"2024-05-08T04:08:27.731781+0000","last_unstale":"2024-05-08T04:13:14.244099+0000","last_undegraded":"2024-05-08T04:13:14.244099+0000","last_fullsized":"2024-05-08T04:13:14.244099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":3951,"log_dups_size":0,"ondisk_log_size":3951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:46:39.944802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":62,"num_read_kb":52,"num_write":3964,"num_write_kb":169350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"23'7381","reported_seq":7401,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.243557+0000","last_change":"2024-05-08T04:08:27.731984+0000","last_active":"2024-05-08T04:13:14.243557+0000","last_peered":"2024-05-08T04:13:14.243557+0000","last_clean":"2024-05-08T04:13:14.243557+0000","last_became_active":"2024-05-08T04:08:27.731598+0000","last_became_peered":"2024-05-08T04:08:27.731598+0000","last_unstale":"2024-05-08T04:13:14.243557+0000","last_undegraded":"2024-05-08T04:13:14.243557+0000","last_fullsized":"2024-05-08T04:13:14.243557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":7381,"log_dups_size":0,"ondisk_log_size":7381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:41:22.212562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":91594752,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":7365,"num_write_kb":213614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"23'5765","reported_seq":5796,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.282559+0000","last_change":"2024-05-08T04:08:27.732284+0000","last_active":"2024-05-08T04:13:14.282559+0000","last_peered":"2024-05-08T04:13:14.282559+0000","last_clean":"2024-05-08T04:13:14.282559+0000","last_became_active":"2024-05-08T04:08:27.732006+0000","last_became_peered":"2024-05-08T04:08:27.732006+0000","last_unstale":"2024-05-08T04:13:14.282559+0000","last_undegraded":"2024-05-08T04:13:14.282559+0000","last_fullsized":"2024-05-08T04:13:14.282559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":5765,"log_dups_size":0,"ondisk_log_size":5765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:24:55.281049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":56,"num_read_kb":44,"num_write":5770,"num_write_kb":177284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"21'4466","reported_seq":4490,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.280830+0000","last_change":"2024-05-08T04:08:27.733083+0000","last_active":"2024-05-08T04:13:14.280830+0000","last_peered":"2024-05-08T04:13:14.280830+0000","last_clean":"2024-05-08T04:13:14.280830+0000","last_became_active":"2024-05-08T04:08:27.732710+0000","last_became_peered":"2024-05-08T04:08:27.732710+0000","last_unstale":"2024-05-08T04:13:14.280830+0000","last_undegraded":"2024-05-08T04:13:14.280830+0000","last_fullsized":"2024-05-08T04:13:14.280830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":4466,"log_dups_size":0,"ondisk_log_size":4466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:40:43.219199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125829120,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":4481,"num_write_kb":186066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"23'4175","reported_seq":4194,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.283656+0000","last_change":"2024-05-08T04:08:27.726522+0000","last_active":"2024-05-08T04:13:14.283656+0000","last_peered":"2024-05-08T04:13:14.283656+0000","last_clean":"2024-05-08T04:13:14.283656+0000","last_became_active":"2024-05-08T04:08:27.726232+0000","last_became_peered":"2024-05-08T04:08:27.726232+0000","last_unstale":"2024-05-08T04:13:14.283656+0000","last_undegraded":"2024-05-08T04:13:14.283656+0000","last_fullsized":"2024-05-08T04:13:14.283656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":4175,"log_dups_size":0,"ondisk_log_size":4175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:07.708671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120203616,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":4136,"num_write_kb":200298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"23'6141","reported_seq":6161,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.246780+0000","last_change":"2024-05-08T04:08:27.722514+0000","last_active":"2024-05-08T04:13:14.246780+0000","last_peered":"2024-05-08T04:13:14.246780+0000","last_clean":"2024-05-08T04:13:14.246780+0000","last_became_active":"2024-05-08T04:08:27.721701+0000","last_became_peered":"2024-05-08T04:08:27.721701+0000","last_unstale":"2024-05-08T04:13:14.246780+0000","last_undegraded":"2024-05-08T04:13:14.246780+0000","last_fullsized":"2024-05-08T04:13:14.246780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":6141,"log_dups_size":0,"ondisk_log_size":6141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:44.554641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":6123,"num_write_kb":209934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"23'5187","reported_seq":5207,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.286850+0000","last_change":"2024-05-08T04:08:27.728824+0000","last_active":"2024-05-08T04:13:14.286850+0000","last_peered":"2024-05-08T04:13:14.286850+0000","last_clean":"2024-05-08T04:13:14.286850+0000","last_became_active":"2024-05-08T04:08:27.728564+0000","last_became_peered":"2024-05-08T04:08:27.728564+0000","last_unstale":"2024-05-08T04:13:14.286850+0000","last_undegraded":"2024-05-08T04:13:14.286850+0000","last_fullsized":"2024-05-08T04:13:14.286850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":5187,"log_dups_size":0,"ondisk_log_size":5187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:55.969304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142606336,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":5146,"num_write_kb":193669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"22'7560","reported_seq":7581,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.243068+0000","last_change":"2024-05-08T04:08:27.728565+0000","last_active":"2024-05-08T04:13:14.243068+0000","last_peered":"2024-05-08T04:13:14.243068+0000","last_clean":"2024-05-08T04:13:14.243068+0000","last_became_active":"2024-05-08T04:08:27.728205+0000","last_became_peered":"2024-05-08T04:08:27.728205+0000","last_unstale":"2024-05-08T04:13:14.243068+0000","last_undegraded":"2024-05-08T04:13:14.243068+0000","last_fullsized":"2024-05-08T04:13:14.243068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":7560,"log_dups_size":0,"ondisk_log_size":7560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:22:53.989163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142606336,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":12,"num_read_kb":135,"num_write":7569,"num_write_kb":191194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"22'5903","reported_seq":5920,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.244254+0000","last_change":"2024-05-08T04:08:27.732104+0000","last_active":"2024-05-08T04:13:14.244254+0000","last_peered":"2024-05-08T04:13:14.244254+0000","last_clean":"2024-05-08T04:13:14.244254+0000","last_became_active":"2024-05-08T04:08:27.731769+0000","last_became_peered":"2024-05-08T04:08:27.731769+0000","last_unstale":"2024-05-08T04:13:14.244254+0000","last_undegraded":"2024-05-08T04:13:14.244254+0000","last_fullsized":"2024-05-08T04:13:14.244254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":5903,"log_dups_size":0,"ondisk_log_size":5903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:21:26.194969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5896,"num_write_kb":160193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"23'4608","reported_seq":4626,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.283448+0000","last_change":"2024-05-08T04:08:27.731045+0000","last_active":"2024-05-08T04:13:14.283448+0000","last_peered":"2024-05-08T04:13:14.283448+0000","last_clean":"2024-05-08T04:13:14.283448+0000","last_became_active":"2024-05-08T04:08:27.730749+0000","last_became_peered":"2024-05-08T04:08:27.730749+0000","last_unstale":"2024-05-08T04:13:14.283448+0000","last_undegraded":"2024-05-08T04:13:14.283448+0000","last_fullsized":"2024-05-08T04:13:14.283448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":4608,"log_dups_size":0,"ondisk_log_size":4608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:17:07.247072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":127271424,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":4592,"num_write_kb":174343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"22'7467","reported_seq":7795,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.243724+0000","last_change":"2024-05-08T04:08:27.729609+0000","last_active":"2024-05-08T04:13:14.243724+0000","last_peered":"2024-05-08T04:13:14.243724+0000","last_clean":"2024-05-08T04:13:14.243724+0000","last_became_active":"2024-05-08T04:08:27.729318+0000","last_became_peered":"2024-05-08T04:08:27.729318+0000","last_unstale":"2024-05-08T04:13:14.243724+0000","last_undegraded":"2024-05-08T04:13:14.243724+0000","last_fullsized":"2024-05-08T04:13:14.243724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":7467,"log_dups_size":0,"ondisk_log_size":7467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:17:45.553221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112068432,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1045,"num_read_kb":996,"num_write":7685,"num_write_kb":175598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"23'6087","reported_seq":6105,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.290448+0000","last_change":"2024-05-08T04:08:27.730319+0000","last_active":"2024-05-08T04:13:14.290448+0000","last_peered":"2024-05-08T04:13:14.290448+0000","last_clean":"2024-05-08T04:13:14.290448+0000","last_became_active":"2024-05-08T04:08:27.729781+0000","last_became_peered":"2024-05-08T04:08:27.729781+0000","last_unstale":"2024-05-08T04:13:14.290448+0000","last_undegraded":"2024-05-08T04:13:14.290448+0000","last_fullsized":"2024-05-08T04:13:14.290448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":6087,"log_dups_size":0,"ondisk_log_size":6087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:29:26.256991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123143848,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6035,"num_write_kb":178933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.280716+0000","last_change":"2024-05-08T04:08:21.697423+0000","last_active":"2024-05-08T04:13:14.280716+0000","last_peered":"2024-05-08T04:13:14.280716+0000","last_clean":"2024-05-08T04:13:14.280716+0000","last_became_active":"2024-05-08T04:08:21.697149+0000","last_became_peered":"2024-05-08T04:08:21.697149+0000","last_unstale":"2024-05-08T04:13:14.280716+0000","last_undegraded":"2024-05-08T04:13:14.280716+0000","last_fullsized":"2024-05-08T04:13:14.280716+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:59:38.354372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'7340","reported_seq":7381,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.244087+0000","last_change":"2024-05-08T04:08:27.725473+0000","last_active":"2024-05-08T04:13:14.244087+0000","last_peered":"2024-05-08T04:13:14.244087+0000","last_clean":"2024-05-08T04:13:14.244087+0000","last_became_active":"2024-05-08T04:08:27.725165+0000","last_became_peered":"2024-05-08T04:08:27.725165+0000","last_unstale":"2024-05-08T04:13:14.244087+0000","last_undegraded":"2024-05-08T04:13:14.244087+0000","last_fullsized":"2024-05-08T04:13:14.244087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":7340,"log_dups_size":0,"ondisk_log_size":7340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:06:19.688474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142738448,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":74,"num_read_kb":59,"num_write":7371,"num_write_kb":187653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.278833+0000","last_change":"2024-05-08T04:08:21.696942+0000","last_active":"2024-05-08T04:13:14.278833+0000","last_peered":"2024-05-08T04:13:14.278833+0000","last_clean":"2024-05-08T04:13:14.278833+0000","last_became_active":"2024-05-08T04:08:21.696632+0000","last_became_peered":"2024-05-08T04:08:21.696632+0000","last_unstale":"2024-05-08T04:13:14.278833+0000","last_undegraded":"2024-05-08T04:13:14.278833+0000","last_fullsized":"2024-05-08T04:13:14.278833+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:23:25.156383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'3712","reported_seq":3730,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.289760+0000","last_change":"2024-05-08T04:08:27.730834+0000","last_active":"2024-05-08T04:13:14.289760+0000","last_peered":"2024-05-08T04:13:14.289760+0000","last_clean":"2024-05-08T04:13:14.289760+0000","last_became_active":"2024-05-08T04:08:27.730530+0000","last_became_peered":"2024-05-08T04:08:27.730530+0000","last_unstale":"2024-05-08T04:13:14.289760+0000","last_undegraded":"2024-05-08T04:13:14.289760+0000","last_fullsized":"2024-05-08T04:13:14.289760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":3712,"log_dups_size":0,"ondisk_log_size":3712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:51:02.242727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3661,"num_write_kb":168963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.244054+0000","last_change":"2024-05-08T04:08:21.696814+0000","last_active":"2024-05-08T04:13:14.244054+0000","last_peered":"2024-05-08T04:13:14.244054+0000","last_clean":"2024-05-08T04:13:14.244054+0000","last_became_active":"2024-05-08T04:08:21.696403+0000","last_became_peered":"2024-05-08T04:08:21.696403+0000","last_unstale":"2024-05-08T04:13:14.244054+0000","last_undegraded":"2024-05-08T04:13:14.244054+0000","last_fullsized":"2024-05-08T04:13:14.244054+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:02:04.075137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"22'6934","reported_seq":6952,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.278367+0000","last_change":"2024-05-08T04:08:27.721621+0000","last_active":"2024-05-08T04:13:14.278367+0000","last_peered":"2024-05-08T04:13:14.278367+0000","last_clean":"2024-05-08T04:13:14.278367+0000","last_became_active":"2024-05-08T04:08:27.721303+0000","last_became_peered":"2024-05-08T04:08:27.721303+0000","last_unstale":"2024-05-08T04:13:14.278367+0000","last_undegraded":"2024-05-08T04:13:14.278367+0000","last_fullsized":"2024-05-08T04:13:14.278367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":6934,"log_dups_size":0,"ondisk_log_size":6934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:59:37.780694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6930,"num_write_kb":193544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.278665+0000","last_change":"2024-05-08T04:08:21.693307+0000","last_active":"2024-05-08T04:13:14.278665+0000","last_peered":"2024-05-08T04:13:14.278665+0000","last_clean":"2024-05-08T04:13:14.278665+0000","last_became_active":"2024-05-08T04:08:21.693014+0000","last_became_peered":"2024-05-08T04:08:21.693014+0000","last_unstale":"2024-05-08T04:13:14.278665+0000","last_undegraded":"2024-05-08T04:13:14.278665+0000","last_fullsized":"2024-05-08T04:13:14.278665+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:56.218680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"22'4514","reported_seq":4533,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.282828+0000","last_change":"2024-05-08T04:08:27.729798+0000","last_active":"2024-05-08T04:13:14.282828+0000","last_peered":"2024-05-08T04:13:14.282828+0000","last_clean":"2024-05-08T04:13:14.282828+0000","last_became_active":"2024-05-08T04:08:27.729504+0000","last_became_peered":"2024-05-08T04:08:27.729504+0000","last_unstale":"2024-05-08T04:13:14.282828+0000","last_undegraded":"2024-05-08T04:13:14.282828+0000","last_fullsized":"2024-05-08T04:13:14.282828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":4514,"log_dups_size":0,"ondisk_log_size":4514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:58:47.867425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96535048,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":4515,"num_write_kb":160844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"21'6272","reported_seq":6403,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.246844+0000","last_change":"2024-05-08T04:08:27.726831+0000","last_active":"2024-05-08T04:13:14.246844+0000","last_peered":"2024-05-08T04:13:14.246844+0000","last_clean":"2024-05-08T04:13:14.246844+0000","last_became_active":"2024-05-08T04:08:27.726527+0000","last_became_peered":"2024-05-08T04:08:27.726527+0000","last_unstale":"2024-05-08T04:13:14.246844+0000","last_undegraded":"2024-05-08T04:13:14.246844+0000","last_fullsized":"2024-05-08T04:13:14.246844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":6272,"log_dups_size":0,"ondisk_log_size":6272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:26:39.243499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":126223376,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":153,"num_read_kb":139,"num_write":6308,"num_write_kb":247140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.280609+0000","last_change":"2024-05-08T04:08:21.699221+0000","last_active":"2024-05-08T04:13:14.280609+0000","last_peered":"2024-05-08T04:13:14.280609+0000","last_clean":"2024-05-08T04:13:14.280609+0000","last_became_active":"2024-05-08T04:08:21.699013+0000","last_became_peered":"2024-05-08T04:08:21.699013+0000","last_unstale":"2024-05-08T04:13:14.280609+0000","last_undegraded":"2024-05-08T04:13:14.280609+0000","last_fullsized":"2024-05-08T04:13:14.280609+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:23:08.040183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"23'3069","reported_seq":3087,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.285582+0000","last_change":"2024-05-08T04:08:27.723409+0000","last_active":"2024-05-08T04:13:14.285582+0000","last_peered":"2024-05-08T04:13:14.285582+0000","last_clean":"2024-05-08T04:13:14.285582+0000","last_became_active":"2024-05-08T04:08:27.723118+0000","last_became_peered":"2024-05-08T04:08:27.723118+0000","last_unstale":"2024-05-08T04:13:14.285582+0000","last_undegraded":"2024-05-08T04:13:14.285582+0000","last_fullsized":"2024-05-08T04:13:14.285582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":3069,"log_dups_size":0,"ondisk_log_size":3069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:31:49.593896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3019,"num_write_kb":152195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-05-08T04:11:58.137838+0000","last_change":"2024-05-08T04:08:21.799283+0000","last_active":"2024-05-08T04:11:58.137838+0000","last_peered":"2024-05-08T04:11:58.137838+0000","last_clean":"2024-05-08T04:11:58.137838+0000","last_became_active":"2024-05-08T04:08:21.798995+0000","last_became_peered":"2024-05-08T04:08:21.798995+0000","last_unstale":"2024-05-08T04:11:58.137838+0000","last_undegraded":"2024-05-08T04:11:58.137838+0000","last_fullsized":"2024-05-08T04:11:58.137838+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:37:16.795888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"23'3229","reported_seq":3248,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.243180+0000","last_change":"2024-05-08T04:08:27.726742+0000","last_active":"2024-05-08T04:13:14.243180+0000","last_peered":"2024-05-08T04:13:14.243180+0000","last_clean":"2024-05-08T04:13:14.243180+0000","last_became_active":"2024-05-08T04:08:27.726457+0000","last_became_peered":"2024-05-08T04:08:27.726457+0000","last_unstale":"2024-05-08T04:13:14.243180+0000","last_undegraded":"2024-05-08T04:13:14.243180+0000","last_fullsized":"2024-05-08T04:13:14.243180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":3229,"log_dups_size":0,"ondisk_log_size":3229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:36:59.353164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104923656,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":3211,"num_write_kb":154644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.243505+0000","last_change":"2024-05-08T04:08:21.697070+0000","last_active":"2024-05-08T04:13:14.243505+0000","last_peered":"2024-05-08T04:13:14.243505+0000","last_clean":"2024-05-08T04:13:14.243505+0000","last_became_active":"2024-05-08T04:08:21.696370+0000","last_became_peered":"2024-05-08T04:08:21.696370+0000","last_unstale":"2024-05-08T04:13:14.243505+0000","last_undegraded":"2024-05-08T04:13:14.243505+0000","last_fullsized":"2024-05-08T04:13:14.243505+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:54.981822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"22'5074","reported_seq":5092,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.243183+0000","last_change":"2024-05-08T04:08:27.732812+0000","last_active":"2024-05-08T04:13:14.243183+0000","last_peered":"2024-05-08T04:13:14.243183+0000","last_clean":"2024-05-08T04:13:14.243183+0000","last_became_active":"2024-05-08T04:08:27.732446+0000","last_became_peered":"2024-05-08T04:08:27.732446+0000","last_unstale":"2024-05-08T04:13:14.243183+0000","last_undegraded":"2024-05-08T04:13:14.243183+0000","last_fullsized":"2024-05-08T04:13:14.243183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":5074,"log_dups_size":0,"ondisk_log_size":5074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:34:38.276980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5070,"num_write_kb":160386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.280726+0000","last_change":"2024-05-08T04:08:21.696942+0000","last_active":"2024-05-08T04:13:14.280726+0000","last_peered":"2024-05-08T04:13:14.280726+0000","last_clean":"2024-05-08T04:13:14.280726+0000","last_became_active":"2024-05-08T04:08:21.696659+0000","last_became_peered":"2024-05-08T04:08:21.696659+0000","last_unstale":"2024-05-08T04:13:14.280726+0000","last_undegraded":"2024-05-08T04:13:14.280726+0000","last_fullsized":"2024-05-08T04:13:14.280726+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:50:44.209520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":116,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.243514+0000","last_change":"2024-05-08T04:08:20.691470+0000","last_active":"2024-05-08T04:13:14.243514+0000","last_peered":"2024-05-08T04:13:14.243514+0000","last_clean":"2024-05-08T04:13:14.243514+0000","last_became_active":"2024-05-08T04:08:20.691204+0000","last_became_peered":"2024-05-08T04:08:20.691204+0000","last_unstale":"2024-05-08T04:13:14.243514+0000","last_undegraded":"2024-05-08T04:13:14.243514+0000","last_fullsized":"2024-05-08T04:13:14.243514+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:19.679313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:19.679313+0000","last_clean_scrub_stamp":"2024-05-08T04:08:19.679313+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:52:31.185659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"23'3940","reported_seq":3963,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.285959+0000","last_change":"2024-05-08T04:08:27.729757+0000","last_active":"2024-05-08T04:13:14.285959+0000","last_peered":"2024-05-08T04:13:14.285959+0000","last_clean":"2024-05-08T04:13:14.285959+0000","last_became_active":"2024-05-08T04:08:27.729321+0000","last_became_peered":"2024-05-08T04:08:27.729321+0000","last_unstale":"2024-05-08T04:13:14.285959+0000","last_undegraded":"2024-05-08T04:13:14.285959+0000","last_fullsized":"2024-05-08T04:13:14.285959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":3940,"log_dups_size":0,"ondisk_log_size":3940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:12:28.413005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109182976,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":3879,"num_write_kb":165315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"23'9391","reported_seq":9539,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.311352+0000","last_change":"2024-05-08T04:08:27.721874+0000","last_active":"2024-05-08T04:13:14.311352+0000","last_peered":"2024-05-08T04:13:14.311352+0000","last_clean":"2024-05-08T04:13:14.311352+0000","last_became_active":"2024-05-08T04:08:27.721534+0000","last_became_peered":"2024-05-08T04:08:27.721534+0000","last_unstale":"2024-05-08T04:13:14.311352+0000","last_undegraded":"2024-05-08T04:13:14.311352+0000","last_fullsized":"2024-05-08T04:13:14.311352+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":9391,"log_dups_size":0,"ondisk_log_size":9391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:59:46.563205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":139724624,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":407,"num_read_kb":368,"num_write":6462,"num_write_kb":194675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"23'3828","reported_seq":3868,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.280742+0000","last_change":"2024-05-08T04:08:27.726164+0000","last_active":"2024-05-08T04:13:14.280742+0000","last_peered":"2024-05-08T04:13:14.280742+0000","last_clean":"2024-05-08T04:13:14.280742+0000","last_became_active":"2024-05-08T04:08:27.725833+0000","last_became_peered":"2024-05-08T04:08:27.725833+0000","last_unstale":"2024-05-08T04:13:14.280742+0000","last_undegraded":"2024-05-08T04:13:14.280742+0000","last_fullsized":"2024-05-08T04:13:14.280742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":3828,"log_dups_size":0,"ondisk_log_size":3828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:35.399549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":85458944,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":91,"num_read_kb":137,"num_write":3837,"num_write_kb":155137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"21'4041","reported_seq":4083,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.243142+0000","last_change":"2024-05-08T04:08:27.727947+0000","last_active":"2024-05-08T04:13:14.243142+0000","last_peered":"2024-05-08T04:13:14.243142+0000","last_clean":"2024-05-08T04:13:14.243142+0000","last_became_active":"2024-05-08T04:08:27.727653+0000","last_became_peered":"2024-05-08T04:08:27.727653+0000","last_unstale":"2024-05-08T04:13:14.243142+0000","last_undegraded":"2024-05-08T04:13:14.243142+0000","last_fullsized":"2024-05-08T04:13:14.243142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":4041,"log_dups_size":0,"ondisk_log_size":4041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:14:37.705194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130023424,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":90,"num_read_kb":141,"num_write":4071,"num_write_kb":155757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"23'5324","reported_seq":5350,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.291432+0000","last_change":"2024-05-08T04:08:27.730956+0000","last_active":"2024-05-08T04:13:14.291432+0000","last_peered":"2024-05-08T04:13:14.291432+0000","last_clean":"2024-05-08T04:13:14.291432+0000","last_became_active":"2024-05-08T04:08:27.730569+0000","last_became_peered":"2024-05-08T04:08:27.730569+0000","last_unstale":"2024-05-08T04:13:14.291432+0000","last_undegraded":"2024-05-08T04:13:14.291432+0000","last_fullsized":"2024-05-08T04:13:14.291432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":5324,"log_dups_size":0,"ondisk_log_size":5324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:05:40.969984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":5289,"num_write_kb":148492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"23'4024","reported_seq":4056,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.243696+0000","last_change":"2024-05-08T04:08:27.733357+0000","last_active":"2024-05-08T04:13:14.243696+0000","last_peered":"2024-05-08T04:13:14.243696+0000","last_clean":"2024-05-08T04:13:14.243696+0000","last_became_active":"2024-05-08T04:08:27.733070+0000","last_became_peered":"2024-05-08T04:08:27.733070+0000","last_unstale":"2024-05-08T04:13:14.243696+0000","last_undegraded":"2024-05-08T04:13:14.243696+0000","last_fullsized":"2024-05-08T04:13:14.243696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":4024,"log_dups_size":0,"ondisk_log_size":4024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:33:08.617696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":103481344,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":53,"num_read_kb":105,"num_write":4128,"num_write_kb":186691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"23'3223","reported_seq":3343,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.298622+0000","last_change":"2024-05-08T04:08:27.730671+0000","last_active":"2024-05-08T04:13:14.298622+0000","last_peered":"2024-05-08T04:13:14.298622+0000","last_clean":"2024-05-08T04:13:14.298622+0000","last_became_active":"2024-05-08T04:08:27.730372+0000","last_became_peered":"2024-05-08T04:08:27.730372+0000","last_unstale":"2024-05-08T04:13:14.298622+0000","last_undegraded":"2024-05-08T04:13:14.298622+0000","last_fullsized":"2024-05-08T04:13:14.298622+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":3223,"log_dups_size":0,"ondisk_log_size":3223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:55:59.085956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":78381056,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":342,"num_read_kb":267,"num_write":3496,"num_write_kb":113772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"23'4349","reported_seq":4370,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.289521+0000","last_change":"2024-05-08T04:08:27.729948+0000","last_active":"2024-05-08T04:13:14.289521+0000","last_peered":"2024-05-08T04:13:14.289521+0000","last_clean":"2024-05-08T04:13:14.289521+0000","last_became_active":"2024-05-08T04:08:27.729584+0000","last_became_peered":"2024-05-08T04:08:27.729584+0000","last_unstale":"2024-05-08T04:13:14.289521+0000","last_undegraded":"2024-05-08T04:13:14.289521+0000","last_fullsized":"2024-05-08T04:13:14.289521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":4349,"log_dups_size":0,"ondisk_log_size":4349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:35:18.017771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95482384,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":4309,"num_write_kb":147779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"23'5306","reported_seq":5332,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.291888+0000","last_change":"2024-05-08T04:08:27.727021+0000","last_active":"2024-05-08T04:13:14.291888+0000","last_peered":"2024-05-08T04:13:14.291888+0000","last_clean":"2024-05-08T04:13:14.291888+0000","last_became_active":"2024-05-08T04:08:27.726692+0000","last_became_peered":"2024-05-08T04:08:27.726692+0000","last_unstale":"2024-05-08T04:13:14.291888+0000","last_undegraded":"2024-05-08T04:13:14.291888+0000","last_fullsized":"2024-05-08T04:13:14.291888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":5306,"log_dups_size":0,"ondisk_log_size":5306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:16:39.270048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120979456,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":5269,"num_write_kb":245378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"21'2450","reported_seq":2468,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.243820+0000","last_change":"2024-05-08T04:08:27.727459+0000","last_active":"2024-05-08T04:13:14.243820+0000","last_peered":"2024-05-08T04:13:14.243820+0000","last_clean":"2024-05-08T04:13:14.243820+0000","last_became_active":"2024-05-08T04:08:27.727138+0000","last_became_peered":"2024-05-08T04:08:27.727138+0000","last_unstale":"2024-05-08T04:13:14.243820+0000","last_undegraded":"2024-05-08T04:13:14.243820+0000","last_fullsized":"2024-05-08T04:13:14.243820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":2450,"log_dups_size":0,"ondisk_log_size":2450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:32:43.624016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2442,"num_write_kb":115332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"23'3790","reported_seq":3816,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.289583+0000","last_change":"2024-05-08T04:08:27.729355+0000","last_active":"2024-05-08T04:13:14.289583+0000","last_peered":"2024-05-08T04:13:14.289583+0000","last_clean":"2024-05-08T04:13:14.289583+0000","last_became_active":"2024-05-08T04:08:27.729024+0000","last_became_peered":"2024-05-08T04:08:27.729024+0000","last_unstale":"2024-05-08T04:13:14.289583+0000","last_undegraded":"2024-05-08T04:13:14.289583+0000","last_fullsized":"2024-05-08T04:13:14.289583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":3790,"log_dups_size":0,"ondisk_log_size":3790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:28:16.629952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96600064,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":3752,"num_write_kb":156932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"23'8760","reported_seq":8780,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.243612+0000","last_change":"2024-05-08T04:08:27.729279+0000","last_active":"2024-05-08T04:13:14.243612+0000","last_peered":"2024-05-08T04:13:14.243612+0000","last_clean":"2024-05-08T04:13:14.243612+0000","last_became_active":"2024-05-08T04:08:27.728925+0000","last_became_peered":"2024-05-08T04:08:27.728925+0000","last_unstale":"2024-05-08T04:13:14.243612+0000","last_undegraded":"2024-05-08T04:13:14.243612+0000","last_fullsized":"2024-05-08T04:13:14.243612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":8760,"log_dups_size":0,"ondisk_log_size":8760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:40:38.725967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163709456,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":8744,"num_write_kb":214668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3581206776,"num_objects":880,"num_object_clones":0,"num_object_copies":1760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":880,"num_whiteouts":0,"num_read":2570,"num_read_kb":2586,"num_write":164525,"num_write_kb":5655773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6823485440,"data_stored":6861063288,"data_compressed":194285506,"data_compressed_allocated":202047488,"data_compressed_original":243892224,"omap_allocated":0,"internal_metadata":0},"log_size":167261,"ondisk_log_size":167261,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":10222,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542206,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1406956,"kb_used_data":1010724,"kb_used_omap":1,"kb_used_meta":396222,"kb_avail":92964884,"statfs":{"total":96636764160,"available":95196041216,"internally_reserved":0,"allocated":1034981376,"data_stored":1042910983,"data_compressed":31421929,"data_compressed_allocated":32739328,"data_compressed_original":41480192,"omap_allocated":1589,"internal_metadata":405731787},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542206,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406912,"kb_used_data":259188,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":93964928,"statfs":{"total":96636764160,"available":96220086272,"internally_reserved":0,"allocated":265408512,"data_stored":266327003,"data_compressed":7220440,"data_compressed_allocated":7479296,"data_compressed_original":8728576,"omap_allocated":1594,"internal_metadata":151255494},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542206,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1269684,"kb_used_data":891496,"kb_used_omap":1,"kb_used_meta":378174,"kb_avail":93102156,"statfs":{"total":96636764160,"available":95336607744,"internally_reserved":0,"allocated":912891904,"data_stored":917912083,"data_compressed":26020264,"data_compressed_allocated":26992640,"data_compressed_original":32751616,"omap_allocated":1595,"internal_metadata":387250629},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542206,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1904428,"kb_used_data":1372192,"kb_used_omap":1,"kb_used_meta":532222,"kb_avail":92467412,"statfs":{"total":96636764160,"available":94686629888,"internally_reserved":0,"allocated":1405124608,"data_stored":1412894735,"data_compressed":41362870,"data_compressed_allocated":43028480,"data_compressed_original":51744768,"omap_allocated":1595,"internal_metadata":544995781},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542206,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1508044,"kb_used_data":1053504,"kb_used_omap":1,"kb_used_meta":454526,"kb_avail":92863796,"statfs":{"total":96636764160,"available":95092527104,"internally_reserved":0,"allocated":1078788096,"data_stored":1080571807,"data_compressed":21885706,"data_compressed_allocated":22720512,"data_compressed_original":25346048,"omap_allocated":1593,"internal_metadata":465435079},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542206,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":952204,"kb_used_data":673988,"kb_used_omap":1,"kb_used_meta":278206,"kb_avail":93419636,"statfs":{"total":96636764160,"available":95661707264,"internally_reserved":0,"allocated":690163712,"data_stored":694964323,"data_compressed":20980979,"data_compressed_allocated":21929984,"data_compressed_original":27267072,"omap_allocated":1595,"internal_metadata":284883397},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542206,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1461556,"kb_used_data":1079208,"kb_used_omap":1,"kb_used_meta":382334,"kb_avail":92910284,"statfs":{"total":96636764160,"available":95140130816,"internally_reserved":0,"allocated":1105108992,"data_stored":1112879471,"data_compressed":36587350,"data_compressed_allocated":38072320,"data_compressed_original":46706688,"omap_allocated":1595,"internal_metadata":391510469},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574910,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":428404,"kb_used_data":325036,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":93943436,"statfs":{"total":96636764160,"available":96198078464,"internally_reserved":0,"allocated":332836864,"data_stored":334240523,"data_compressed":8816190,"data_compressed_allocated":9150464,"data_compressed_original":10891264,"omap_allocated":1595,"internal_metadata":105839045},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5111,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5111,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1104838656,"data_stored":1112277372,"data_compressed":36582239,"data_compressed_allocated":38039552,"data_compressed_original":46194688,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":689958912,"data_stored":694895920,"data_compressed":20980979,"data_compressed_allocated":21929984,"data_compressed_original":27267072,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":332632064,"data_stored":334172120,"data_compressed":8816190,"data_compressed_allocated":9150464,"data_compressed_original":10891264,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1078571008,"data_stored":1080498636,"data_compressed":21885706,"data_compressed_allocated":22720512,"data_compressed_original":25346048,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1404907520,"data_stored":1412821564,"data_compressed":41362870,"data_compressed_allocated":43028480,"data_compressed_original":51744768,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":912674816,"data_stored":917838912,"data_compressed":26020264,"data_compressed_allocated":26992640,"data_compressed_original":32751616,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":265191424,"data_stored":266253832,"data_compressed":7220440,"data_compressed_allocated":7479296,"data_compressed_original":8728576,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1034711040,"data_stored":1042304932,"data_compressed":31416818,"data_compressed_allocated":32706560,"data_compressed_original":40968192,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:13:15.749 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T04:13:15.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.845+0000 7f4f04c7e640 1 Processor -- start 2024-05-08T04:13:15.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.845+0000 7f4f04c7e640 1 -- start start 2024-05-08T04:13:15.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.846+0000 7f4f04c7e640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4f00143a30 0x7f4f00072bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:15.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.846+0000 7f4f04c7e640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f000b7230 con 0x7f4f000b6e50 2024-05-08T04:13:15.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.846+0000 7f4f04c7e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f00074950 con 0x7f4f00143ea0 2024-05-08T04:13:15.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.846+0000 7f4f04c7e640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7f4f001442e0 con 0x7f4f00143a30 2024-05-08T04:13:15.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.846+0000 7f4efed9d640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4f00143a30 0x7f4f00072bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:15.848 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.846+0000 7f4efe59c640 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f4f000b6e50 0x7f4f00147ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43724/0 (socket says 172.21.15.170:43724) 2024-05-08T04:13:15.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.846+0000 7f4efe59c640 1 -- 172.21.15.170:0/645228944 learned_addr learned my addr 172.21.15.170:0/645228944 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-05-08T04:13:15.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.846+0000 7f4efed9d640 1 -- 172.21.15.170:0/645228944 >> v1:172.21.15.187:6789/0 conn(0x7f4f00143ea0 legacy=0x7f4f00143320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:15.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.846+0000 7f4efed9d640 1 -- 172.21.15.170:0/645228944 >> v1:172.21.15.170:6789/0 conn(0x7f4f000b6e50 legacy=0x7f4f00147ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:15.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.846+0000 7f4efed9d640 1 -- 172.21.15.170:0/645228944 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f001475c0 con 0x7f4f00143a30 2024-05-08T04:13:15.849 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.846+0000 7f4efdd9b640 1 -- 172.21.15.170:0/645228944 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 855712434 0 0) 0x7f4f000b7230 con 0x7f4f000b6e50 2024-05-08T04:13:15.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.847+0000 7f4efed9d640 1 --2- 172.21.15.170:0/645228944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f00143a30 0x7f4f00072bc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8009d90 tx=0x7f4ef802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd54eeb2b0b66ebc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:15.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.847+0000 7f4efdd9b640 1 -- 172.21.15.170:0/645228944 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8039070 con 0x7f4f00143a30 2024-05-08T04:13:15.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.847+0000 7f4efdd9b640 1 -- 172.21.15.170:0/645228944 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ef802cd00 con 0x7f4f00143a30 2024-05-08T04:13:15.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.847+0000 7f4efdd9b640 1 -- 172.21.15.170:0/645228944 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8034660 con 0x7f4f00143a30 2024-05-08T04:13:15.850 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.847+0000 7f4f04c7e640 1 -- 172.21.15.170:0/645228944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f00143a30 msgr2=0x7f4f00072bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:15.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.847+0000 7f4f04c7e640 1 --2- 172.21.15.170:0/645228944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f00143a30 0x7f4f00072bc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8009d90 tx=0x7f4ef802bf50 comp rx=0 tx=0).stop 2024-05-08T04:13:15.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.848+0000 7f4f04c7e640 1 -- 172.21.15.170:0/645228944 shutdown_connections 2024-05-08T04:13:15.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.848+0000 7f4f04c7e640 1 --2- 172.21.15.170:0/645228944 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f00143a30 0x7f4f00072bc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:15.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.848+0000 7f4f04c7e640 1 -- 172.21.15.170:0/645228944 >> 172.21.15.170:0/645228944 conn(0x7f4f000ba000 msgr2=0x7f4f000b64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:15.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.848+0000 7f4f04c7e640 1 -- 172.21.15.170:0/645228944 shutdown_connections 2024-05-08T04:13:15.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.848+0000 7f4f04c7e640 1 -- 172.21.15.170:0/645228944 wait complete. 2024-05-08T04:13:15.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.849+0000 7f4f04c7e640 1 Processor -- start 2024-05-08T04:13:15.853 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.849+0000 7f4f04c7e640 1 -- start start 2024-05-08T04:13:15.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.849+0000 7f4f04c7e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f000b6e50 0x7f4f00146cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:15.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.849+0000 7f4f04c7e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f00143a30 0x7f4f001395c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:15.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.849+0000 7f4f04c7e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4f00143ea0 0x7f4f000b8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:15.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.849+0000 7f4f04c7e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4f00073eb0 con 0x7f4f000b6e50 2024-05-08T04:13:15.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.849+0000 7f4f04c7e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4f00073d30 con 0x7f4f00143a30 2024-05-08T04:13:15.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.849+0000 7f4f04c7e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f4f0014f7c0 con 0x7f4f00143ea0 2024-05-08T04:13:15.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.850+0000 7f4efe59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f000b6e50 0x7f4f00146cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:15.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.850+0000 7f4efe59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f000b6e50 0x7f4f00146cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49336/0 (socket says 172.21.15.170:49336) 2024-05-08T04:13:15.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.850+0000 7f4efe59c640 1 -- 172.21.15.170:0/639546115 learned_addr learned my addr 172.21.15.170:0/639546115 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:13:15.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.850+0000 7f4efed9d640 1 --2- 172.21.15.170:0/639546115 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4f00143ea0 0x7f4f000b8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:15.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.850+0000 7f4ef5d9b640 1 --2- 172.21.15.170:0/639546115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f00143a30 0x7f4f001395c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:15.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.850+0000 7f4efe59c640 1 -- 172.21.15.170:0/639546115 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4f00143ea0 msgr2=0x7f4f000b8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:15.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.850+0000 7f4efe59c640 1 --2- 172.21.15.170:0/639546115 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4f00143ea0 0x7f4f000b8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:15.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.850+0000 7f4efe59c640 1 -- 172.21.15.170:0/639546115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f00143a30 msgr2=0x7f4f001395c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:15.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.850+0000 7f4efe59c640 1 --2- 172.21.15.170:0/639546115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f00143a30 0x7f4f001395c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:15.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.850+0000 7f4efe59c640 1 -- 172.21.15.170:0/639546115 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f000b8960 con 0x7f4f000b6e50 2024-05-08T04:13:15.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.850+0000 7f4efed9d640 1 --2- 172.21.15.170:0/639546115 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4f00143ea0 0x7f4f000b8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:13:15.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.850+0000 7f4ef5d9b640 1 --2- 172.21.15.170:0/639546115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f00143a30 0x7f4f001395c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:13:15.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.850+0000 7f4efe59c640 1 --2- 172.21.15.170:0/639546115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f000b6e50 0x7f4f00146cb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4ef000c450 tx=0x7f4ef000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:15.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.850+0000 7f4ef77fe640 1 -- 172.21.15.170:0/639546115 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0018070 con 0x7f4f000b6e50 2024-05-08T04:13:15.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.850+0000 7f4ef77fe640 1 -- 172.21.15.170:0/639546115 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ef00074e0 con 0x7f4f000b6e50 2024-05-08T04:13:15.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.850+0000 7f4ef77fe640 1 -- 172.21.15.170:0/639546115 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0004a20 con 0x7f4f000b6e50 2024-05-08T04:13:15.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.850+0000 7f4f04c7e640 1 -- 172.21.15.170:0/639546115 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f00138dc0 con 0x7f4f000b6e50 2024-05-08T04:13:15.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.850+0000 7f4f04c7e640 1 -- 172.21.15.170:0/639546115 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f00137b60 con 0x7f4f000b6e50 2024-05-08T04:13:15.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.851+0000 7f4f04c7e640 1 -- 172.21.15.170:0/639546115 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f0006e9e0 con 0x7f4f000b6e50 2024-05-08T04:13:15.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.852+0000 7f4ef77fe640 1 -- 172.21.15.170:0/639546115 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f4ef0004bc0 con 0x7f4f000b6e50 2024-05-08T04:13:15.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.852+0000 7f4ef77fe640 1 --2- 172.21.15.170:0/639546115 >> v2:172.21.15.187:6816/101007897 conn(0x7f4ee00762c0 0x7f4ee0078780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:15.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.852+0000 7f4ef77fe640 1 -- 172.21.15.170:0/639546115 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 4525+0+0 (secure 0 0 0) 0x7f4ef0096200 con 0x7f4f000b6e50 2024-05-08T04:13:15.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.853+0000 7f4ef5d9b640 1 --2- 172.21.15.170:0/639546115 >> v2:172.21.15.187:6816/101007897 conn(0x7f4ee00762c0 0x7f4ee0078780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:15.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.853+0000 7f4ef5d9b640 1 --2- 172.21.15.170:0/639546115 >> v2:172.21.15.187:6816/101007897 conn(0x7f4ee00762c0 0x7f4ee0078780 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8007fd0 tx=0x7f4ee8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:15.858 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:15.857+0000 7f4ef77fe640 1 -- 172.21.15.170:0/639546115 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f4ef005fb00 con 0x7f4f000b6e50 2024-05-08T04:13:16.040 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:16.038+0000 7f4f04c7e640 1 -- 172.21.15.170:0/639546115 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4f00071580 con 0x7f4f000b6e50 2024-05-08T04:13:16.041 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:16.039+0000 7f4ef77fe640 1 -- 172.21.15.170:0/639546115 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+14698 (secure 0 0 0) 0x7f4ef0068140 con 0x7f4f000b6e50 2024-05-08T04:13:16.041 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:13:16.041 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":23,"fsid":"e5497bb3-afa8-4f24-8ef4-42911ffcc962","created":"2024-05-08T04:08:09.270498+0000","modified":"2024-05-08T04:13:14.235759+0000","last_up_change":"2024-05-08T04:08:18.673075+0000","last_in_change":"2024-05-08T04:08:11.562526+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T04:08:19.670334+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-08T04:08:20.197207+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephsqlite","create_time":"2024-05-08T04:08:26.676023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6bca968b-d825-417a-b907-5312aad8dbcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":1037021465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6801","nonce":1037021465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6803","nonce":1037021465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":1037021465}]},"public_addr":"172.21.15.170:6800/1037021465","cluster_addr":"172.21.15.170:6801/1037021465","heartbeat_back_addr":"172.21.15.170:6803/1037021465","heartbeat_front_addr":"172.21.15.170:6802/1037021465","state":["exists","up"]},{"osd":1,"uuid":"66600f89-4532-43b1-85d3-6d8c736cfaf2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3364584304}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6809","nonce":3364584304}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6811","nonce":3364584304}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3364584304}]},"public_addr":"172.21.15.170:6808/3364584304","cluster_addr":"172.21.15.170:6809/3364584304","heartbeat_back_addr":"172.21.15.170:6811/3364584304","heartbeat_front_addr":"172.21.15.170:6810/3364584304","state":["exists","up"]},{"osd":2,"uuid":"a11608b2-0871-497c-aa38-264baea6bf39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":3333432230}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6805","nonce":3333432230}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6807","nonce":3333432230}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":3333432230}]},"public_addr":"172.21.15.170:6804/3333432230","cluster_addr":"172.21.15.170:6805/3333432230","heartbeat_back_addr":"172.21.15.170:6807/3333432230","heartbeat_front_addr":"172.21.15.170:6806/3333432230","state":["exists","up"]},{"osd":3,"uuid":"8caae6f8-790c-426c-af8a-3a9ef54eae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":153555120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6813","nonce":153555120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6815","nonce":153555120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":153555120}]},"public_addr":"172.21.15.170:6812/153555120","cluster_addr":"172.21.15.170:6813/153555120","heartbeat_back_addr":"172.21.15.170:6815/153555120","heartbeat_front_addr":"172.21.15.170:6814/153555120","state":["exists","up"]},{"osd":4,"uuid":"0e804208-3607-4d71-b771-f79fb27e7968","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":1425849822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":1425849822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":1425849822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":1425849822}]},"public_addr":"172.21.15.187:6808/1425849822","cluster_addr":"172.21.15.187:6811/1425849822","heartbeat_back_addr":"172.21.15.187:6815/1425849822","heartbeat_front_addr":"172.21.15.187:6813/1425849822","state":["exists","up"]},{"osd":5,"uuid":"f88f88db-216e-4acc-ba5a-89d91b98aeb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":3273856499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":3273856499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6803","nonce":3273856499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":3273856499}]},"public_addr":"172.21.15.187:6800/3273856499","cluster_addr":"172.21.15.187:6801/3273856499","heartbeat_back_addr":"172.21.15.187:6803/3273856499","heartbeat_front_addr":"172.21.15.187:6802/3273856499","state":["exists","up"]},{"osd":6,"uuid":"6b6840fc-1e34-4cc8-9da6-8a57e69f319b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":2911331731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":2911331731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":2911331731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":2911331731}]},"public_addr":"172.21.15.187:6809/2911331731","cluster_addr":"172.21.15.187:6810/2911331731","heartbeat_back_addr":"172.21.15.187:6814/2911331731","heartbeat_front_addr":"172.21.15.187:6812/2911331731","state":["exists","up"]},{"osd":7,"uuid":"5dbcf804-181b-4d5b-9a40-f99ddaee2701","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":2815417961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":2815417961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6807","nonce":2815417961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":2815417961}]},"public_addr":"172.21.15.187:6804/2815417961","cluster_addr":"172.21.15.187:6805/2815417961","heartbeat_back_addr":"172.21.15.187:6807/2815417961","heartbeat_front_addr":"172.21.15.187:6806/2815417961","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:08:16.738630+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:08:16.681089+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:08:16.630258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:08:16.670263+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:08:16.737547+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:08:16.673870+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:08:16.706810+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T04:08:16.658743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.170:0/4031121872":"2024-05-08T05:13:13.322295+0000","172.21.15.170:0/70137236":"2024-05-08T05:11:57.167485+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T04:13:16.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:16.042+0000 7f4f04c7e640 1 -- 172.21.15.170:0/639546115 >> v2:172.21.15.187:6816/101007897 conn(0x7f4ee00762c0 msgr2=0x7f4ee0078780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:16.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:16.042+0000 7f4f04c7e640 1 --2- 172.21.15.170:0/639546115 >> v2:172.21.15.187:6816/101007897 conn(0x7f4ee00762c0 0x7f4ee0078780 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8007fd0 tx=0x7f4ee8009040 comp rx=0 tx=0).stop 2024-05-08T04:13:16.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:16.042+0000 7f4f04c7e640 1 -- 172.21.15.170:0/639546115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f000b6e50 msgr2=0x7f4f00146cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:16.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:16.042+0000 7f4f04c7e640 1 --2- 172.21.15.170:0/639546115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f000b6e50 0x7f4f00146cb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4ef000c450 tx=0x7f4ef000c910 comp rx=0 tx=0).stop 2024-05-08T04:13:16.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:16.042+0000 7f4f04c7e640 1 -- 172.21.15.170:0/639546115 shutdown_connections 2024-05-08T04:13:16.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:16.042+0000 7f4f04c7e640 1 --2- 172.21.15.170:0/639546115 >> v2:172.21.15.187:6816/101007897 conn(0x7f4ee00762c0 0x7f4ee0078780 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:16.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:16.042+0000 7f4f04c7e640 1 --2- 172.21.15.170:0/639546115 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4f00143a30 0x7f4f001395c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:16.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:16.042+0000 7f4f04c7e640 1 --2- 172.21.15.170:0/639546115 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4f00143ea0 0x7f4f000b8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:16.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:16.042+0000 7f4f04c7e640 1 --2- 172.21.15.170:0/639546115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4f000b6e50 0x7f4f00146cb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:16.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:16.042+0000 7f4f04c7e640 1 -- 172.21.15.170:0/639546115 >> 172.21.15.170:0/639546115 conn(0x7f4f000ba000 msgr2=0x7f4f001430f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:16.044 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:16.042+0000 7f4f04c7e640 1 -- 172.21.15.170:0/639546115 shutdown_connections 2024-05-08T04:13:16.045 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:16.042+0000 7f4f04c7e640 1 -- 172.21.15.170:0/639546115 wait complete. 2024-05-08T04:13:17.056 INFO:tasks.ceph:Scrubbing osd.0 2024-05-08T04:13:17.056 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-08T04:13:17.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.169+0000 7fde2cee6640 1 Processor -- start 2024-05-08T04:13:17.171 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.169+0000 7fde2cee6640 1 -- start start 2024-05-08T04:13:17.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.170+0000 7fde2cee6640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fde28147880 0x7fde281433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:17.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.170+0000 7fde2cee6640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fde280712c0 0x7fde2814e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:17.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.170+0000 7fde2cee6640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde280b7470 con 0x7fde28143900 2024-05-08T04:13:17.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.170+0000 7fde2cee6640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fde281484f0 con 0x7fde280712c0 2024-05-08T04:13:17.172 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.170+0000 7fde2cee6640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fde28147fd0 con 0x7fde28147880 2024-05-08T04:13:17.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.170+0000 7fde2659c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fde28147880 0x7fde281433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:17.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.170+0000 7fde2659c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fde28147880 0x7fde281433c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:47974/0 (socket says 172.21.15.170:47974) 2024-05-08T04:13:17.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.170+0000 7fde2659c640 1 -- 172.21.15.170:0/3527580375 learned_addr learned my addr 172.21.15.170:0/3527580375 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:13:17.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.170+0000 7fde26d9d640 1 --2- 172.21.15.170:0/3527580375 >> v2:172.21.15.187:3300/0 conn(0x7fde280712c0 0x7fde2814e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:17.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.170+0000 7fde2659c640 1 -- 172.21.15.170:0/3527580375 >> v2:172.21.15.187:3300/0 conn(0x7fde280712c0 msgr2=0x7fde2814e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:13:17.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.170+0000 7fde2659c640 1 --2- 172.21.15.170:0/3527580375 >> v2:172.21.15.187:3300/0 conn(0x7fde280712c0 0x7fde2814e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.170+0000 7fde2659c640 1 -- 172.21.15.170:0/3527580375 >> v1:172.21.15.187:6789/0 conn(0x7fde28143900 legacy=0x7fde28070bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:13:17.173 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.170+0000 7fde2659c640 1 -- 172.21.15.170:0/3527580375 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde28147c60 con 0x7fde28147880 2024-05-08T04:13:17.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.171+0000 7fde2659c640 1 --2- 172.21.15.170:0/3527580375 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fde28147880 0x7fde281433c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fde18009e50 tx=0x7fde1802be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33a889adad2c4cf0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:17.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.171+0000 7fde2559a640 1 -- 172.21.15.170:0/3527580375 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1802c9e0 con 0x7fde28147880 2024-05-08T04:13:17.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.171+0000 7fde2559a640 1 -- 172.21.15.170:0/3527580375 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde1802cb80 con 0x7fde28147880 2024-05-08T04:13:17.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.171+0000 7fde2559a640 1 -- 172.21.15.170:0/3527580375 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde180343f0 con 0x7fde28147880 2024-05-08T04:13:17.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.172+0000 7fde2cee6640 1 -- 172.21.15.170:0/3527580375 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fde28147880 msgr2=0x7fde281433c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:17.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.172+0000 7fde2cee6640 1 --2- 172.21.15.170:0/3527580375 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fde28147880 0x7fde281433c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fde18009e50 tx=0x7fde1802be40 comp rx=0 tx=0).stop 2024-05-08T04:13:17.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.172+0000 7fde2cee6640 1 -- 172.21.15.170:0/3527580375 shutdown_connections 2024-05-08T04:13:17.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.172+0000 7fde2cee6640 1 --2- 172.21.15.170:0/3527580375 >> v2:172.21.15.187:3300/0 conn(0x7fde280712c0 0x7fde2814e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.172+0000 7fde2cee6640 1 --2- 172.21.15.170:0/3527580375 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fde28147880 0x7fde281433c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.172+0000 7fde2cee6640 1 -- 172.21.15.170:0/3527580375 >> 172.21.15.170:0/3527580375 conn(0x7fde280b6570 msgr2=0x7fde280b6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:17.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.172+0000 7fde2cee6640 1 -- 172.21.15.170:0/3527580375 shutdown_connections 2024-05-08T04:13:17.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.172+0000 7fde2cee6640 1 -- 172.21.15.170:0/3527580375 wait complete. 2024-05-08T04:13:17.175 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.173+0000 7fde2cee6640 1 Processor -- start 2024-05-08T04:13:17.176 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.173+0000 7fde2cee6640 1 -- start start 2024-05-08T04:13:17.177 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.174+0000 7fde2cee6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde280712c0 0x7fde280b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:17.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.174+0000 7fde2cee6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde28143900 0x7fde280afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:17.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.174+0000 7fde2cee6640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fde28147880 0x7fde2813a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:17.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.174+0000 7fde2cee6640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fde281450a0 con 0x7fde280712c0 2024-05-08T04:13:17.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.174+0000 7fde2cee6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fde28071800 con 0x7fde28143900 2024-05-08T04:13:17.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.174+0000 7fde2cee6640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fde28145220 con 0x7fde28147880 2024-05-08T04:13:17.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.174+0000 7fde26d9d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fde28147880 0x7fde2813a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:17.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.174+0000 7fde2659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde280712c0 0x7fde280b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:17.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.174+0000 7fde26d9d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fde28147880 0x7fde2813a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:47990/0 (socket says 172.21.15.170:47990) 2024-05-08T04:13:17.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.174+0000 7fde26d9d640 1 -- 172.21.15.170:0/3025296081 learned_addr learned my addr 172.21.15.170:0/3025296081 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:13:17.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.174+0000 7fde25d9b640 1 --2- 172.21.15.170:0/3025296081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde28143900 0x7fde280afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:17.178 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.174+0000 7fde26d9d640 1 -- 172.21.15.170:0/3025296081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde28143900 msgr2=0x7fde280afe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:17.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.174+0000 7fde26d9d640 1 --2- 172.21.15.170:0/3025296081 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde28143900 0x7fde280afe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.174+0000 7fde26d9d640 1 -- 172.21.15.170:0/3025296081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde280712c0 msgr2=0x7fde280b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:17.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.174+0000 7fde26d9d640 1 --2- 172.21.15.170:0/3025296081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde280712c0 0x7fde280b0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.174+0000 7fde26d9d640 1 -- 172.21.15.170:0/3025296081 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde280b0340 con 0x7fde28147880 2024-05-08T04:13:17.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.174+0000 7fde2659c640 1 --2- 172.21.15.170:0/3025296081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde280712c0 0x7fde280b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:13:17.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.174+0000 7fde26d9d640 1 --2- 172.21.15.170:0/3025296081 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fde28147880 0x7fde2813a070 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fde2000c820 tx=0x7fde2000cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:17.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.174+0000 7fde16ffd640 1 -- 172.21.15.170:0/3025296081 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde20015070 con 0x7fde28147880 2024-05-08T04:13:17.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.175+0000 7fde16ffd640 1 -- 172.21.15.170:0/3025296081 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde20004020 con 0x7fde28147880 2024-05-08T04:13:17.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.175+0000 7fde2cee6640 1 -- 172.21.15.170:0/3025296081 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde280af430 con 0x7fde28147880 2024-05-08T04:13:17.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.175+0000 7fde16ffd640 1 -- 172.21.15.170:0/3025296081 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde20004410 con 0x7fde28147880 2024-05-08T04:13:17.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.175+0000 7fde2cee6640 1 -- 172.21.15.170:0/3025296081 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fde2813ae80 con 0x7fde28147880 2024-05-08T04:13:17.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.175+0000 7fde2cee6640 1 -- 172.21.15.170:0/3025296081 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fde2806d400 con 0x7fde28147880 2024-05-08T04:13:17.179 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.176+0000 7fde16ffd640 1 -- 172.21.15.170:0/3025296081 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7fde20007830 con 0x7fde28147880 2024-05-08T04:13:17.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.177+0000 7fde16ffd640 1 --2- 172.21.15.170:0/3025296081 >> v2:172.21.15.187:6816/101007897 conn(0x7fde08076290 0x7fde08078750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:17.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.177+0000 7fde16ffd640 1 -- 172.21.15.170:0/3025296081 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 4525+0+0 (secure 0 0 0) 0x7fde20096da0 con 0x7fde28147880 2024-05-08T04:13:17.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.177+0000 7fde2659c640 1 --2- 172.21.15.170:0/3025296081 >> v2:172.21.15.187:6816/101007897 conn(0x7fde08076290 0x7fde08078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:17.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.177+0000 7fde16ffd640 1 --2- 172.21.15.170:0/3025296081 >> v2:172.21.15.170:6800/1037021465 conn(0x7fde0807cff0 0x7fde0807f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:17.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.177+0000 7fde16ffd640 1 -- 172.21.15.170:0/3025296081 --> v2:172.21.15.170:6800/1037021465 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fde0807fa80 con 0x7fde0807cff0 2024-05-08T04:13:17.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.177+0000 7fde16ffd640 1 -- 172.21.15.170:0/3025296081 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fde200994a0 con 0x7fde28147880 2024-05-08T04:13:17.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.177+0000 7fde25d9b640 1 --2- 172.21.15.170:0/3025296081 >> v2:172.21.15.170:6800/1037021465 conn(0x7fde0807cff0 0x7fde0807f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:17.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.177+0000 7fde25d9b640 1 --2- 172.21.15.170:0/3025296081 >> v2:172.21.15.170:6800/1037021465 conn(0x7fde0807cff0 0x7fde0807f410 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:17.181 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.177+0000 7fde2659c640 1 --2- 172.21.15.170:0/3025296081 >> v2:172.21.15.187:6816/101007897 conn(0x7fde08076290 0x7fde08078750 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fde18035000 tx=0x7fde18030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:17.182 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.178+0000 7fde16ffd640 1 -- 172.21.15.170:0/3025296081 <== osd.0 v2:172.21.15.170:6800/1037021465 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fde0807fa80 con 0x7fde0807cff0 2024-05-08T04:13:17.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.191+0000 7fde2cee6640 1 -- 172.21.15.170:0/3025296081 --> v2:172.21.15.170:6800/1037021465 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fde2813a860 con 0x7fde0807cff0 2024-05-08T04:13:17.193 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.191+0000 7fde16ffd640 1 -- 172.21.15.170:0/3025296081 <== osd.0 v2:172.21.15.170:6800/1037021465 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fde2813a860 con 0x7fde0807cff0 2024-05-08T04:13:17.193 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-05-08T04:13:17.193 INFO:teuthology.orchestra.run.smithi170.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T04:13:17.193 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-05-08T04:13:17.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.191+0000 7fde2cee6640 1 -- 172.21.15.170:0/3025296081 >> v2:172.21.15.170:6800/1037021465 conn(0x7fde0807cff0 msgr2=0x7fde0807f410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:17.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.191+0000 7fde2cee6640 1 --2- 172.21.15.170:0/3025296081 >> v2:172.21.15.170:6800/1037021465 conn(0x7fde0807cff0 0x7fde0807f410 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.192+0000 7fde2cee6640 1 -- 172.21.15.170:0/3025296081 >> v2:172.21.15.187:6816/101007897 conn(0x7fde08076290 msgr2=0x7fde08078750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:17.194 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.192+0000 7fde2cee6640 1 --2- 172.21.15.170:0/3025296081 >> v2:172.21.15.187:6816/101007897 conn(0x7fde08076290 0x7fde08078750 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fde18035000 tx=0x7fde18030000 comp rx=0 tx=0).stop 2024-05-08T04:13:17.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.192+0000 7fde2cee6640 1 -- 172.21.15.170:0/3025296081 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fde28147880 msgr2=0x7fde2813a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:17.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.192+0000 7fde2cee6640 1 --2- 172.21.15.170:0/3025296081 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fde28147880 0x7fde2813a070 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fde2000c820 tx=0x7fde2000cce0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.192+0000 7fde26d9d640 1 -- 172.21.15.170:0/3025296081 reap_dead start 2024-05-08T04:13:17.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.192+0000 7fde2cee6640 1 -- 172.21.15.170:0/3025296081 shutdown_connections 2024-05-08T04:13:17.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.192+0000 7fde2cee6640 1 -- 172.21.15.170:0/3025296081 >> 172.21.15.170:0/3025296081 conn(0x7fde280b6570 msgr2=0x7fde28072b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:17.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.192+0000 7fde2cee6640 1 -- 172.21.15.170:0/3025296081 shutdown_connections 2024-05-08T04:13:17.195 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.192+0000 7fde2cee6640 1 -- 172.21.15.170:0/3025296081 wait complete. 2024-05-08T04:13:17.205 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-08T04:13:17.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.305+0000 7f8cfb666640 1 Processor -- start 2024-05-08T04:13:17.307 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.305+0000 7f8cfb666640 1 -- start start 2024-05-08T04:13:17.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.306+0000 7f8cfb666640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8cf40b6e50 0x7f8cf4143ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:17.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.306+0000 7f8cfb666640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8cf4143320 0x7f8cf4143750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:17.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.306+0000 7f8cfb666640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cf40b7230 con 0x7f8cf4072bc0 2024-05-08T04:13:17.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.306+0000 7f8cfb666640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7f8cf4143c90 con 0x7f8cf40b6e50 2024-05-08T04:13:17.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.306+0000 7f8cfb666640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f8cf4074950 con 0x7f8cf4143320 2024-05-08T04:13:17.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.306+0000 7f8cf8c01640 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f8cf4072bc0 0x7f8cf4147ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43736/0 (socket says 172.21.15.170:43736) 2024-05-08T04:13:17.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.306+0000 7f8cf8c01640 1 -- 172.21.15.170:0/1057733739 learned_addr learned my addr 172.21.15.170:0/1057733739 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-05-08T04:13:17.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.306+0000 7f8cf9402640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8cf40b6e50 0x7f8cf4143ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:17.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.306+0000 7f8cf9c03640 1 --2- 172.21.15.170:0/1057733739 >> v2:172.21.15.187:3300/0 conn(0x7f8cf4143320 0x7f8cf4143750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:17.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.306+0000 7f8cf9402640 1 -- 172.21.15.170:0/1057733739 >> v2:172.21.15.187:3300/0 conn(0x7f8cf4143320 msgr2=0x7f8cf4143750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:13:17.309 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.306+0000 7f8cf9402640 1 --2- 172.21.15.170:0/1057733739 >> v2:172.21.15.187:3300/0 conn(0x7f8cf4143320 0x7f8cf4143750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.306+0000 7f8cf9402640 1 -- 172.21.15.170:0/1057733739 >> v1:172.21.15.170:6789/0 conn(0x7f8cf4072bc0 legacy=0x7f8cf4147ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:17.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.306+0000 7f8cebfff640 1 -- 172.21.15.170:0/1057733739 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 109252109 0 0) 0x7f8cf40b7230 con 0x7f8cf4072bc0 2024-05-08T04:13:17.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.306+0000 7f8cf9402640 1 -- 172.21.15.170:0/1057733739 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf41475c0 con 0x7f8cf40b6e50 2024-05-08T04:13:17.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.307+0000 7f8cf9402640 1 --2- 172.21.15.170:0/1057733739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cf40b6e50 0x7f8cf4143ea0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8cf406a040 tx=0x7f8cf002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61cbfe9ba86ad93 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:17.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.307+0000 7f8cebfff640 1 -- 172.21.15.170:0/1057733739 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0035020 con 0x7f8cf40b6e50 2024-05-08T04:13:17.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.307+0000 7f8cebfff640 1 -- 172.21.15.170:0/1057733739 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cf0031830 con 0x7f8cf40b6e50 2024-05-08T04:13:17.310 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.307+0000 7f8cebfff640 1 -- 172.21.15.170:0/1057733739 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0031c00 con 0x7f8cf40b6e50 2024-05-08T04:13:17.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.308+0000 7f8cfb666640 1 -- 172.21.15.170:0/1057733739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cf40b6e50 msgr2=0x7f8cf4143ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:17.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.308+0000 7f8cfb666640 1 --2- 172.21.15.170:0/1057733739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cf40b6e50 0x7f8cf4143ea0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8cf406a040 tx=0x7f8cf002b8d0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.308+0000 7f8cfb666640 1 -- 172.21.15.170:0/1057733739 shutdown_connections 2024-05-08T04:13:17.313 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.308+0000 7f8cfb666640 1 --2- 172.21.15.170:0/1057733739 >> v2:172.21.15.187:3300/0 conn(0x7f8cf4143320 0x7f8cf4143750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.308+0000 7f8cfb666640 1 --2- 172.21.15.170:0/1057733739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cf40b6e50 0x7f8cf4143ea0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.308+0000 7f8cfb666640 1 -- 172.21.15.170:0/1057733739 >> 172.21.15.170:0/1057733739 conn(0x7f8cf40ba000 msgr2=0x7f8cf40b64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:17.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.308+0000 7f8cfb666640 1 -- 172.21.15.170:0/1057733739 shutdown_connections 2024-05-08T04:13:17.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.308+0000 7f8cfb666640 1 -- 172.21.15.170:0/1057733739 wait complete. 2024-05-08T04:13:17.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.309+0000 7f8cfb666640 1 Processor -- start 2024-05-08T04:13:17.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.309+0000 7f8cfb666640 1 -- start start 2024-05-08T04:13:17.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.310+0000 7f8cfb666640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cf4072bc0 0x7f8cf40b9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:17.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.310+0000 7f8cfb666640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8cf40b6e50 0x7f8cf413ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:17.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.310+0000 7f8cfb666640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cf4143320 0x7f8cf413c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:17.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.310+0000 7f8cfb666640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf4148370 con 0x7f8cf4072bc0 2024-05-08T04:13:17.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.310+0000 7f8cfb666640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf41481f0 con 0x7f8cf4143320 2024-05-08T04:13:17.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.310+0000 7f8cfb666640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f8cf4073d30 con 0x7f8cf40b6e50 2024-05-08T04:13:17.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.310+0000 7f8cf9402640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cf4072bc0 0x7f8cf40b9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:17.314 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.310+0000 7f8cf8c01640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8cf40b6e50 0x7f8cf413ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:17.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.310+0000 7f8cf9402640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cf4072bc0 0x7f8cf40b9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49362/0 (socket says 172.21.15.170:49362) 2024-05-08T04:13:17.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.310+0000 7f8cf9402640 1 -- 172.21.15.170:0/209592178 learned_addr learned my addr 172.21.15.170:0/209592178 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:13:17.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.310+0000 7f8cf9c03640 1 --2- 172.21.15.170:0/209592178 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cf4143320 0x7f8cf413c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:17.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.310+0000 7f8cf9402640 1 -- 172.21.15.170:0/209592178 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8cf40b6e50 msgr2=0x7f8cf413ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:17.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.310+0000 7f8cf9402640 1 --2- 172.21.15.170:0/209592178 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8cf40b6e50 0x7f8cf413ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.310+0000 7f8cf9402640 1 -- 172.21.15.170:0/209592178 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cf4143320 msgr2=0x7f8cf413c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:17.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.310+0000 7f8cf9402640 1 --2- 172.21.15.170:0/209592178 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cf4143320 0x7f8cf413c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.310+0000 7f8cf9402640 1 -- 172.21.15.170:0/209592178 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf413e160 con 0x7f8cf4072bc0 2024-05-08T04:13:17.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.310+0000 7f8cf8c01640 1 --2- 172.21.15.170:0/209592178 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8cf40b6e50 0x7f8cf413ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:13:17.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.310+0000 7f8cf9402640 1 --2- 172.21.15.170:0/209592178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cf4072bc0 0x7f8cf40b9100 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0000c00 tx=0x7f8cf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:17.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.310+0000 7f8ce9ffb640 1 -- 172.21.15.170:0/209592178 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0035070 con 0x7f8cf4072bc0 2024-05-08T04:13:17.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.310+0000 7f8ce9ffb640 1 -- 172.21.15.170:0/209592178 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cf0004250 con 0x7f8cf4072bc0 2024-05-08T04:13:17.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.310+0000 7f8cfb666640 1 -- 172.21.15.170:0/209592178 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf413e440 con 0x7f8cf4072bc0 2024-05-08T04:13:17.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.310+0000 7f8ce9ffb640 1 -- 172.21.15.170:0/209592178 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cf003a6f0 con 0x7f8cf4072bc0 2024-05-08T04:13:17.315 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.311+0000 7f8cfb666640 1 -- 172.21.15.170:0/209592178 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf413c710 con 0x7f8cf4072bc0 2024-05-08T04:13:17.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.311+0000 7f8cfb666640 1 -- 172.21.15.170:0/209592178 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cf406e9e0 con 0x7f8cf4072bc0 2024-05-08T04:13:17.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.312+0000 7f8ce9ffb640 1 -- 172.21.15.170:0/209592178 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f8cf0038020 con 0x7f8cf4072bc0 2024-05-08T04:13:17.316 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.312+0000 7f8ce9ffb640 1 --2- 172.21.15.170:0/209592178 >> v2:172.21.15.187:6816/101007897 conn(0x7f8cd80762c0 0x7f8cd8078780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:17.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.313+0000 7f8ce9ffb640 1 -- 172.21.15.170:0/209592178 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 4525+0+0 (secure 0 0 0) 0x7f8cf00b8be0 con 0x7f8cf4072bc0 2024-05-08T04:13:17.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.313+0000 7f8cf8c01640 1 --2- 172.21.15.170:0/209592178 >> v2:172.21.15.187:6816/101007897 conn(0x7f8cd80762c0 0x7f8cd8078780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:17.317 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.313+0000 7f8cf8c01640 1 --2- 172.21.15.170:0/209592178 >> v2:172.21.15.187:6816/101007897 conn(0x7f8cd80762c0 0x7f8cd8078780 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8cf40b9830 tx=0x7f8ce4007cf0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:17.318 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.317+0000 7f8ce9ffb640 1 -- 172.21.15.170:0/209592178 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f8cf0040070 con 0x7f8cf4072bc0 2024-05-08T04:13:17.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.497+0000 7f8cfb666640 1 -- 172.21.15.170:0/209592178 --> v2:172.21.15.187:6816/101007897 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f8cf4136ee0 con 0x7f8cd80762c0 2024-05-08T04:13:17.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.497+0000 7f8ce9ffb640 1 -- 172.21.15.170:0/209592178 <== mgr.4107 v2:172.21.15.187:6816/101007897 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8cf4136ee0 con 0x7f8cd80762c0 2024-05-08T04:13:17.499 INFO:teuthology.orchestra.run.smithi170.stderr:instructed osd(s) 0 to deep-scrub 2024-05-08T04:13:17.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.499+0000 7f8cfb666640 1 -- 172.21.15.170:0/209592178 >> v2:172.21.15.187:6816/101007897 conn(0x7f8cd80762c0 msgr2=0x7f8cd8078780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:17.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.499+0000 7f8cfb666640 1 --2- 172.21.15.170:0/209592178 >> v2:172.21.15.187:6816/101007897 conn(0x7f8cd80762c0 0x7f8cd8078780 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8cf40b9830 tx=0x7f8ce4007cf0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.499+0000 7f8cfb666640 1 -- 172.21.15.170:0/209592178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cf4072bc0 msgr2=0x7f8cf40b9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:17.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.499+0000 7f8cfb666640 1 --2- 172.21.15.170:0/209592178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cf4072bc0 0x7f8cf40b9100 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0000c00 tx=0x7f8cf00046c0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.500+0000 7f8cfb666640 1 -- 172.21.15.170:0/209592178 shutdown_connections 2024-05-08T04:13:17.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.500+0000 7f8cfb666640 1 --2- 172.21.15.170:0/209592178 >> v2:172.21.15.187:6816/101007897 conn(0x7f8cd80762c0 0x7f8cd8078780 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.500+0000 7f8cfb666640 1 --2- 172.21.15.170:0/209592178 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8cf4143320 0x7f8cf413c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.500+0000 7f8cfb666640 1 --2- 172.21.15.170:0/209592178 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8cf40b6e50 0x7f8cf413ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.500+0000 7f8cfb666640 1 --2- 172.21.15.170:0/209592178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cf4072bc0 0x7f8cf40b9100 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.500+0000 7f8cfb666640 1 -- 172.21.15.170:0/209592178 >> 172.21.15.170:0/209592178 conn(0x7f8cf40ba000 msgr2=0x7f8cf413b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:17.502 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.500+0000 7f8cfb666640 1 -- 172.21.15.170:0/209592178 shutdown_connections 2024-05-08T04:13:17.503 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.500+0000 7f8cfb666640 1 -- 172.21.15.170:0/209592178 wait complete. 2024-05-08T04:13:17.513 INFO:tasks.ceph:Scrubbing osd.1 2024-05-08T04:13:17.513 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-08T04:13:17.616 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.615+0000 7ff65e10a640 1 Processor -- start 2024-05-08T04:13:17.617 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.615+0000 7ff65e10a640 1 -- start start 2024-05-08T04:13:17.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.615+0000 7ff65e10a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff658138900 0x7ff6580aac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:17.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.615+0000 7ff65e10a640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff6581429d0 0x7ff6580ab150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:17.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.615+0000 7ff65e10a640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff658142240 con 0x7ff658072e50 2024-05-08T04:13:17.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.615+0000 7ff65e10a640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7ff6580acb50 con 0x7ff6581429d0 2024-05-08T04:13:17.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.615+0000 7ff65e10a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff6581424b0 con 0x7ff658138900 2024-05-08T04:13:17.619 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.615+0000 7ff64ffff640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff6581429d0 0x7ff6580ab150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:17.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.615+0000 7ff657fff640 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7ff658072e50 0x7ff658145310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43750/0 (socket says 172.21.15.170:43750) 2024-05-08T04:13:17.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.615+0000 7ff64ffff640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff6581429d0 0x7ff6580ab150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49366/0 (socket says 172.21.15.170:49366) 2024-05-08T04:13:17.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.615+0000 7ff657fff640 1 -- 172.21.15.170:0/2334758102 learned_addr learned my addr 172.21.15.170:0/2334758102 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-05-08T04:13:17.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.616+0000 7ff6577fe640 1 --2- 172.21.15.170:0/2334758102 >> v2:172.21.15.187:3300/0 conn(0x7ff658138900 0x7ff6580aac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:17.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.616+0000 7ff64ffff640 1 -- 172.21.15.170:0/2334758102 >> v2:172.21.15.187:3300/0 conn(0x7ff658138900 msgr2=0x7ff6580aac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:13:17.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.616+0000 7ff64ffff640 1 --2- 172.21.15.170:0/2334758102 >> v2:172.21.15.187:3300/0 conn(0x7ff658138900 0x7ff6580aac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.616+0000 7ff64ffff640 1 -- 172.21.15.170:0/2334758102 >> v1:172.21.15.170:6789/0 conn(0x7ff658072e50 legacy=0x7ff658145310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:17.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.616+0000 7ff64ffff640 1 -- 172.21.15.170:0/2334758102 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff658068f50 con 0x7ff6581429d0 2024-05-08T04:13:17.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.616+0000 7ff656ffd640 1 -- 172.21.15.170:0/2334758102 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3375371278 0 0) 0x7ff658142240 con 0x7ff658072e50 2024-05-08T04:13:17.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.616+0000 7ff64ffff640 1 --2- 172.21.15.170:0/2334758102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6581429d0 0x7ff6580ab150 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff6400097a0 tx=0x7ff64002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43db91cc791516d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:17.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.617+0000 7ff656ffd640 1 -- 172.21.15.170:0/2334758102 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff640035020 con 0x7ff6581429d0 2024-05-08T04:13:17.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.617+0000 7ff656ffd640 1 -- 172.21.15.170:0/2334758102 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff640031830 con 0x7ff6581429d0 2024-05-08T04:13:17.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.617+0000 7ff656ffd640 1 -- 172.21.15.170:0/2334758102 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff640031c00 con 0x7ff6581429d0 2024-05-08T04:13:17.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.617+0000 7ff65e10a640 1 -- 172.21.15.170:0/2334758102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6581429d0 msgr2=0x7ff6580ab150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:17.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.617+0000 7ff65e10a640 1 --2- 172.21.15.170:0/2334758102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6581429d0 0x7ff6580ab150 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff6400097a0 tx=0x7ff64002b8d0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.617+0000 7ff65e10a640 1 -- 172.21.15.170:0/2334758102 shutdown_connections 2024-05-08T04:13:17.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.617+0000 7ff65e10a640 1 --2- 172.21.15.170:0/2334758102 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff6581429d0 0x7ff6580ab150 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.617+0000 7ff65e10a640 1 --2- 172.21.15.170:0/2334758102 >> v2:172.21.15.187:3300/0 conn(0x7ff658138900 0x7ff6580aac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.617+0000 7ff65e10a640 1 -- 172.21.15.170:0/2334758102 >> 172.21.15.170:0/2334758102 conn(0x7ff6580b4420 msgr2=0x7ff6580b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:17.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.617+0000 7ff65e10a640 1 -- 172.21.15.170:0/2334758102 shutdown_connections 2024-05-08T04:13:17.621 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.617+0000 7ff65e10a640 1 -- 172.21.15.170:0/2334758102 wait complete. 2024-05-08T04:13:17.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.618+0000 7ff65e10a640 1 Processor -- start 2024-05-08T04:13:17.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.618+0000 7ff65e10a640 1 -- start start 2024-05-08T04:13:17.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.619+0000 7ff65e10a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff658072e50 0x7ff6580b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:17.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.619+0000 7ff65e10a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff658138900 0x7ff6580b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:17.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.619+0000 7ff65e10a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff6581429d0 0x7ff6581d9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:17.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.619+0000 7ff65e10a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff658073480 con 0x7ff658072e50 2024-05-08T04:13:17.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.619+0000 7ff65e10a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff658073300 con 0x7ff658138900 2024-05-08T04:13:17.625 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.619+0000 7ff65e10a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7ff658073600 con 0x7ff6581429d0 2024-05-08T04:13:17.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.619+0000 7ff6577fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff658072e50 0x7ff6580b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:17.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.619+0000 7ff6577fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff658072e50 0x7ff6580b1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49376/0 (socket says 172.21.15.170:49376) 2024-05-08T04:13:17.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.619+0000 7ff6577fe640 1 -- 172.21.15.170:0/1379453926 learned_addr learned my addr 172.21.15.170:0/1379453926 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:13:17.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.619+0000 7ff657fff640 1 --2- 172.21.15.170:0/1379453926 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff6581429d0 0x7ff6581d9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:17.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.619+0000 7ff64ffff640 1 --2- 172.21.15.170:0/1379453926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff658138900 0x7ff6580b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:17.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.619+0000 7ff6577fe640 1 -- 172.21.15.170:0/1379453926 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff6581429d0 msgr2=0x7ff6581d9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:17.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.619+0000 7ff6577fe640 1 --2- 172.21.15.170:0/1379453926 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff6581429d0 0x7ff6581d9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.619+0000 7ff6577fe640 1 -- 172.21.15.170:0/1379453926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff658138900 msgr2=0x7ff6580b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:17.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.619+0000 7ff6577fe640 1 --2- 172.21.15.170:0/1379453926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff658138900 0x7ff6580b2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.619+0000 7ff6577fe640 1 -- 172.21.15.170:0/1379453926 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff658138ce0 con 0x7ff658072e50 2024-05-08T04:13:17.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.619+0000 7ff6577fe640 1 --2- 172.21.15.170:0/1379453926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff658072e50 0x7ff6580b1c20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff64800e9b0 tx=0x7ff64800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:17.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.620+0000 7ff654ff9640 1 -- 172.21.15.170:0/1379453926 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff648017070 con 0x7ff658072e50 2024-05-08T04:13:17.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.620+0000 7ff654ff9640 1 -- 172.21.15.170:0/1379453926 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff64800c9a0 con 0x7ff658072e50 2024-05-08T04:13:17.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.620+0000 7ff654ff9640 1 -- 172.21.15.170:0/1379453926 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64800cd90 con 0x7ff658072e50 2024-05-08T04:13:17.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.620+0000 7ff65e10a640 1 -- 172.21.15.170:0/1379453926 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6580b2300 con 0x7ff658072e50 2024-05-08T04:13:17.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.620+0000 7ff65e10a640 1 -- 172.21.15.170:0/1379453926 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6581d9b40 con 0x7ff658072e50 2024-05-08T04:13:17.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.620+0000 7ff65e10a640 1 -- 172.21.15.170:0/1379453926 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff65806e9e0 con 0x7ff658072e50 2024-05-08T04:13:17.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.621+0000 7ff654ff9640 1 -- 172.21.15.170:0/1379453926 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7ff64801a070 con 0x7ff658072e50 2024-05-08T04:13:17.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.622+0000 7ff654ff9640 1 --2- 172.21.15.170:0/1379453926 >> v2:172.21.15.187:6816/101007897 conn(0x7ff638076290 0x7ff638078750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:17.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.622+0000 7ff654ff9640 1 -- 172.21.15.170:0/1379453926 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 4525+0+0 (secure 0 0 0) 0x7ff648097080 con 0x7ff658072e50 2024-05-08T04:13:17.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.622+0000 7ff654ff9640 1 --2- 172.21.15.170:0/1379453926 >> v2:172.21.15.170:6808/3364584304 conn(0x7ff63807cff0 0x7ff63807f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:17.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.622+0000 7ff64ffff640 1 --2- 172.21.15.170:0/1379453926 >> v2:172.21.15.187:6816/101007897 conn(0x7ff638076290 0x7ff638078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:17.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.622+0000 7ff654ff9640 1 -- 172.21.15.170:0/1379453926 --> v2:172.21.15.170:6808/3364584304 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff648021ec0 con 0x7ff63807cff0 2024-05-08T04:13:17.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.622+0000 7ff654ff9640 1 -- 172.21.15.170:0/1379453926 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff64809e050 con 0x7ff658072e50 2024-05-08T04:13:17.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.622+0000 7ff657fff640 1 --2- 172.21.15.170:0/1379453926 >> v2:172.21.15.170:6808/3364584304 conn(0x7ff63807cff0 0x7ff63807f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:17.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.622+0000 7ff64ffff640 1 --2- 172.21.15.170:0/1379453926 >> v2:172.21.15.187:6816/101007897 conn(0x7ff638076290 0x7ff638078750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff640036000 tx=0x7ff64003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:17.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.622+0000 7ff657fff640 1 --2- 172.21.15.170:0/1379453926 >> v2:172.21.15.170:6808/3364584304 conn(0x7ff63807cff0 0x7ff63807f410 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:17.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.623+0000 7ff654ff9640 1 -- 172.21.15.170:0/1379453926 <== osd.1 v2:172.21.15.170:6808/3364584304 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff648021ec0 con 0x7ff63807cff0 2024-05-08T04:13:17.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.636+0000 7ff65e10a640 1 -- 172.21.15.170:0/1379453926 --> v2:172.21.15.170:6808/3364584304 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff6580ab7b0 con 0x7ff63807cff0 2024-05-08T04:13:17.637 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.636+0000 7ff654ff9640 1 -- 172.21.15.170:0/1379453926 <== osd.1 v2:172.21.15.170:6808/3364584304 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff6580ab7b0 con 0x7ff63807cff0 2024-05-08T04:13:17.638 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-05-08T04:13:17.638 INFO:teuthology.orchestra.run.smithi170.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T04:13:17.638 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-05-08T04:13:17.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.636+0000 7ff65e10a640 1 -- 172.21.15.170:0/1379453926 >> v2:172.21.15.170:6808/3364584304 conn(0x7ff63807cff0 msgr2=0x7ff63807f410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:17.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.636+0000 7ff65e10a640 1 --2- 172.21.15.170:0/1379453926 >> v2:172.21.15.170:6808/3364584304 conn(0x7ff63807cff0 0x7ff63807f410 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.636+0000 7ff65e10a640 1 -- 172.21.15.170:0/1379453926 >> v2:172.21.15.187:6816/101007897 conn(0x7ff638076290 msgr2=0x7ff638078750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:17.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.636+0000 7ff65e10a640 1 --2- 172.21.15.170:0/1379453926 >> v2:172.21.15.187:6816/101007897 conn(0x7ff638076290 0x7ff638078750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff640036000 tx=0x7ff64003c040 comp rx=0 tx=0).stop 2024-05-08T04:13:17.639 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.636+0000 7ff65e10a640 1 -- 172.21.15.170:0/1379453926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff658072e50 msgr2=0x7ff6580b1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:17.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.636+0000 7ff65e10a640 1 --2- 172.21.15.170:0/1379453926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff658072e50 0x7ff6580b1c20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff64800e9b0 tx=0x7ff64800ee70 comp rx=0 tx=0).stop 2024-05-08T04:13:17.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.636+0000 7ff657fff640 1 -- 172.21.15.170:0/1379453926 reap_dead start 2024-05-08T04:13:17.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.636+0000 7ff65e10a640 1 -- 172.21.15.170:0/1379453926 shutdown_connections 2024-05-08T04:13:17.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.636+0000 7ff65e10a640 1 -- 172.21.15.170:0/1379453926 >> 172.21.15.170:0/1379453926 conn(0x7ff6580b4420 msgr2=0x7ff65813f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:17.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.636+0000 7ff65e10a640 1 -- 172.21.15.170:0/1379453926 shutdown_connections 2024-05-08T04:13:17.640 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.636+0000 7ff65e10a640 1 -- 172.21.15.170:0/1379453926 wait complete. 2024-05-08T04:13:17.649 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-08T04:13:17.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.748+0000 7fa1d7fe8640 1 Processor -- start 2024-05-08T04:13:17.750 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.748+0000 7fa1d7fe8640 1 -- start start 2024-05-08T04:13:17.752 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.749+0000 7fa1d7fe8640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa1d00728d0 0x7fa1d0070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:17.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.749+0000 7fa1d7fe8640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa1d013aa20 0x7fa1d01445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:17.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.749+0000 7fa1d7fe8640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1d0073020 con 0x7fa1d0073950 2024-05-08T04:13:17.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.749+0000 7fa1d7fe8640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa1d00aecd0 con 0x7fa1d00728d0 2024-05-08T04:13:17.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.749+0000 7fa1d7fe8640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fa1d00710f0 con 0x7fa1d013aa20 2024-05-08T04:13:17.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.749+0000 7fa1d5583640 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fa1d0073950 0x7fa1d013a310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43756/0 (socket says 172.21.15.170:43756) 2024-05-08T04:13:17.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.749+0000 7fa1d6585640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa1d013aa20 0x7fa1d01445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:17.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.749+0000 7fa1d5583640 1 -- 172.21.15.170:0/2944878222 learned_addr learned my addr 172.21.15.170:0/2944878222 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-05-08T04:13:17.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.749+0000 7fa1d5d84640 1 --2- 172.21.15.170:0/2944878222 >> v2:172.21.15.187:3300/0 conn(0x7fa1d00728d0 0x7fa1d0070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:17.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.749+0000 7fa1d4d82640 1 -- 172.21.15.170:0/2944878222 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3970559160 0 0) 0x7fa1d0073020 con 0x7fa1d0073950 2024-05-08T04:13:17.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.749+0000 7fa1d4d82640 1 -- 172.21.15.170:0/2944878222 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1bc003520 con 0x7fa1d0073950 2024-05-08T04:13:17.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.749+0000 7fa1d6585640 1 -- 172.21.15.170:0/2944878222 >> v2:172.21.15.187:3300/0 conn(0x7fa1d00728d0 msgr2=0x7fa1d0070bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:13:17.753 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.749+0000 7fa1d6585640 1 --2- 172.21.15.170:0/2944878222 >> v2:172.21.15.187:3300/0 conn(0x7fa1d00728d0 0x7fa1d0070bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.749+0000 7fa1d6585640 1 -- 172.21.15.170:0/2944878222 >> v1:172.21.15.170:6789/0 conn(0x7fa1d0073950 legacy=0x7fa1d013a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:17.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.749+0000 7fa1d6585640 1 -- 172.21.15.170:0/2944878222 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d0073d90 con 0x7fa1d013aa20 2024-05-08T04:13:17.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.750+0000 7fa1d6585640 1 --2- 172.21.15.170:0/2944878222 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa1d013aa20 0x7fa1d01445a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009d90 tx=0x7fa1c802bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97709555e2f893be server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:17.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.750+0000 7fa1d4d82640 1 -- 172.21.15.170:0/2944878222 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8039070 con 0x7fa1d013aa20 2024-05-08T04:13:17.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.750+0000 7fa1d4d82640 1 -- 172.21.15.170:0/2944878222 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c802cc50 con 0x7fa1d013aa20 2024-05-08T04:13:17.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.750+0000 7fa1d4d82640 1 -- 172.21.15.170:0/2944878222 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8034440 con 0x7fa1d013aa20 2024-05-08T04:13:17.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.750+0000 7fa1d7fe8640 1 -- 172.21.15.170:0/2944878222 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa1d013aa20 msgr2=0x7fa1d01445a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:17.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.750+0000 7fa1d7fe8640 1 --2- 172.21.15.170:0/2944878222 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa1d013aa20 0x7fa1d01445a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8009d90 tx=0x7fa1c802bea0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.750+0000 7fa1d7fe8640 1 -- 172.21.15.170:0/2944878222 shutdown_connections 2024-05-08T04:13:17.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.750+0000 7fa1d7fe8640 1 --2- 172.21.15.170:0/2944878222 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa1d013aa20 0x7fa1d01445a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.750+0000 7fa1d7fe8640 1 --2- 172.21.15.170:0/2944878222 >> v2:172.21.15.187:3300/0 conn(0x7fa1d00728d0 0x7fa1d0070bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.750+0000 7fa1d7fe8640 1 -- 172.21.15.170:0/2944878222 >> 172.21.15.170:0/2944878222 conn(0x7fa1d00b6710 msgr2=0x7fa1d00b6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:17.754 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.751+0000 7fa1d7fe8640 1 -- 172.21.15.170:0/2944878222 shutdown_connections 2024-05-08T04:13:17.755 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.751+0000 7fa1d7fe8640 1 -- 172.21.15.170:0/2944878222 wait complete. 2024-05-08T04:13:17.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.752+0000 7fa1d7fe8640 1 Processor -- start 2024-05-08T04:13:17.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.752+0000 7fa1d7fe8640 1 -- start start 2024-05-08T04:13:17.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.752+0000 7fa1d7fe8640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa1d00728d0 0x7fa1d013b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:17.757 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.752+0000 7fa1d7fe8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1d0073950 0x7fa1d012fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:17.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.752+0000 7fa1d7fe8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1d013aa20 0x7fa1d013ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:17.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.752+0000 7fa1d7fe8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d0144c60 con 0x7fa1d013aa20 2024-05-08T04:13:17.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.752+0000 7fa1d7fe8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d0144ae0 con 0x7fa1d0073950 2024-05-08T04:13:17.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.752+0000 7fa1d7fe8640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fa1d0144de0 con 0x7fa1d00728d0 2024-05-08T04:13:17.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.752+0000 7fa1d5d84640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa1d00728d0 0x7fa1d013b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:17.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.752+0000 7fa1d6585640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1d013aa20 0x7fa1d013ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:17.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.752+0000 7fa1d5d84640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa1d00728d0 0x7fa1d013b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:48020/0 (socket says 172.21.15.170:48020) 2024-05-08T04:13:17.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.752+0000 7fa1d6585640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1d013aa20 0x7fa1d013ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49384/0 (socket says 172.21.15.170:49384) 2024-05-08T04:13:17.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.752+0000 7fa1d5d84640 1 -- 172.21.15.170:0/3682084076 learned_addr learned my addr 172.21.15.170:0/3682084076 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:13:17.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.752+0000 7fa1d5583640 1 --2- 172.21.15.170:0/3682084076 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1d0073950 0x7fa1d012fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:17.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.752+0000 7fa1d5d84640 1 -- 172.21.15.170:0/3682084076 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1d0073950 msgr2=0x7fa1d012fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:17.758 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.752+0000 7fa1d5d84640 1 --2- 172.21.15.170:0/3682084076 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1d0073950 0x7fa1d012fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.752+0000 7fa1d5d84640 1 -- 172.21.15.170:0/3682084076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1d013aa20 msgr2=0x7fa1d013ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:17.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.752+0000 7fa1d5d84640 1 --2- 172.21.15.170:0/3682084076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1d013aa20 0x7fa1d013ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.752+0000 7fa1d5d84640 1 -- 172.21.15.170:0/3682084076 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d0130340 con 0x7fa1d00728d0 2024-05-08T04:13:17.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.753+0000 7fa1d5d84640 1 --2- 172.21.15.170:0/3682084076 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa1d00728d0 0x7fa1d013b2b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc00e9e0 tx=0x7fa1cc00eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:17.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.753+0000 7fa1c67fc640 1 -- 172.21.15.170:0/3682084076 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc017070 con 0x7fa1d00728d0 2024-05-08T04:13:17.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.753+0000 7fa1c67fc640 1 -- 172.21.15.170:0/3682084076 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1cc00c9d0 con 0x7fa1d00728d0 2024-05-08T04:13:17.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.753+0000 7fa1d7fe8640 1 -- 172.21.15.170:0/3682084076 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d013d350 con 0x7fa1d00728d0 2024-05-08T04:13:17.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.753+0000 7fa1c67fc640 1 -- 172.21.15.170:0/3682084076 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc00cdc0 con 0x7fa1d00728d0 2024-05-08T04:13:17.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.753+0000 7fa1d7fe8640 1 -- 172.21.15.170:0/3682084076 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d01435d0 con 0x7fa1d00728d0 2024-05-08T04:13:17.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.754+0000 7fa1d7fe8640 1 -- 172.21.15.170:0/3682084076 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1d00ae960 con 0x7fa1d00728d0 2024-05-08T04:13:17.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.755+0000 7fa1c67fc640 1 -- 172.21.15.170:0/3682084076 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7fa1cc01a070 con 0x7fa1d00728d0 2024-05-08T04:13:17.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.756+0000 7fa1c67fc640 1 --2- 172.21.15.170:0/3682084076 >> v2:172.21.15.187:6816/101007897 conn(0x7fa1b40762c0 0x7fa1b4078780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:17.759 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.756+0000 7fa1c67fc640 1 -- 172.21.15.170:0/3682084076 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 4525+0+0 (secure 0 0 0) 0x7fa1cc097000 con 0x7fa1d00728d0 2024-05-08T04:13:17.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.756+0000 7fa1d5583640 1 --2- 172.21.15.170:0/3682084076 >> v2:172.21.15.187:6816/101007897 conn(0x7fa1b40762c0 0x7fa1b4078780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:17.760 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.756+0000 7fa1d5583640 1 --2- 172.21.15.170:0/3682084076 >> v2:172.21.15.187:6816/101007897 conn(0x7fa1b40762c0 0x7fa1b4078780 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0007a60 tx=0x7fa1c0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:17.761 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.760+0000 7fa1c67fc640 1 -- 172.21.15.170:0/3682084076 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa1d00ae960 con 0x7fa1d00728d0 2024-05-08T04:13:17.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.982+0000 7fa1d7fe8640 1 -- 172.21.15.170:0/3682084076 --> v2:172.21.15.187:6816/101007897 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fa1d006e9e0 con 0x7fa1b40762c0 2024-05-08T04:13:17.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.983+0000 7fa1c67fc640 1 -- 172.21.15.170:0/3682084076 <== mgr.4107 v2:172.21.15.187:6816/101007897 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa1d006e9e0 con 0x7fa1b40762c0 2024-05-08T04:13:17.985 INFO:teuthology.orchestra.run.smithi170.stderr:instructed osd(s) 1 to deep-scrub 2024-05-08T04:13:17.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.986+0000 7fa1d7fe8640 1 -- 172.21.15.170:0/3682084076 >> v2:172.21.15.187:6816/101007897 conn(0x7fa1b40762c0 msgr2=0x7fa1b4078780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:17.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.986+0000 7fa1d7fe8640 1 --2- 172.21.15.170:0/3682084076 >> v2:172.21.15.187:6816/101007897 conn(0x7fa1b40762c0 0x7fa1b4078780 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0007a60 tx=0x7fa1c0009040 comp rx=0 tx=0).stop 2024-05-08T04:13:17.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.986+0000 7fa1d7fe8640 1 -- 172.21.15.170:0/3682084076 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa1d00728d0 msgr2=0x7fa1d013b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:17.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.986+0000 7fa1d7fe8640 1 --2- 172.21.15.170:0/3682084076 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa1d00728d0 0x7fa1d013b2b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc00e9e0 tx=0x7fa1cc00eea0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.986+0000 7fa1d7fe8640 1 -- 172.21.15.170:0/3682084076 shutdown_connections 2024-05-08T04:13:17.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.986+0000 7fa1d7fe8640 1 --2- 172.21.15.170:0/3682084076 >> v2:172.21.15.187:6816/101007897 conn(0x7fa1b40762c0 0x7fa1b4078780 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.986+0000 7fa1d7fe8640 1 --2- 172.21.15.170:0/3682084076 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1d0073950 0x7fa1d012fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.986+0000 7fa1d7fe8640 1 --2- 172.21.15.170:0/3682084076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1d013aa20 0x7fa1d013ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.986+0000 7fa1d7fe8640 1 --2- 172.21.15.170:0/3682084076 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fa1d00728d0 0x7fa1d013b2b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:17.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.986+0000 7fa1d7fe8640 1 -- 172.21.15.170:0/3682084076 >> 172.21.15.170:0/3682084076 conn(0x7fa1d00b6710 msgr2=0x7fa1d01393d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:17.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.986+0000 7fa1d7fe8640 1 -- 172.21.15.170:0/3682084076 shutdown_connections 2024-05-08T04:13:17.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:17.986+0000 7fa1d7fe8640 1 -- 172.21.15.170:0/3682084076 wait complete. 2024-05-08T04:13:18.000 INFO:tasks.ceph:Scrubbing osd.2 2024-05-08T04:13:18.000 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-08T04:13:18.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.113+0000 7f1fa856a640 1 Processor -- start 2024-05-08T04:13:18.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.113+0000 7f1fa856a640 1 -- start start 2024-05-08T04:13:18.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.114+0000 7f1fa856a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1fa006f580 0x7f1fa00724c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:18.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.114+0000 7f1fa856a640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f1fa0144bc0 0x7f1fa013d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:18.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.114+0000 7f1fa856a640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fa0144420 con 0x7f1fa00720e0 2024-05-08T04:13:18.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.114+0000 7f1fa856a640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7f1fa00aedd0 con 0x7f1fa0144bc0 2024-05-08T04:13:18.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.114+0000 7f1fa856a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1fa0073100 con 0x7f1fa006f580 2024-05-08T04:13:18.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.114+0000 7f1fa6b07640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f1fa0144bc0 0x7f1fa013d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:18.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.114+0000 7f1fa6306640 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f1fa00720e0 0x7f1fa006ee70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43758/0 (socket says 172.21.15.170:43758) 2024-05-08T04:13:18.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.114+0000 7f1fa6306640 1 -- 172.21.15.170:0/1763447761 learned_addr learned my addr 172.21.15.170:0/1763447761 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-05-08T04:13:18.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.114+0000 7f1fa5b05640 1 --2- 172.21.15.170:0/1763447761 >> v2:172.21.15.187:3300/0 conn(0x7f1fa006f580 0x7f1fa00724c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:18.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.114+0000 7f1fa6b07640 1 -- 172.21.15.170:0/1763447761 >> v2:172.21.15.187:3300/0 conn(0x7f1fa006f580 msgr2=0x7f1fa00724c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:13:18.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.114+0000 7f1fa6b07640 1 --2- 172.21.15.170:0/1763447761 >> v2:172.21.15.187:3300/0 conn(0x7f1fa006f580 0x7f1fa00724c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:18.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.114+0000 7f1fa6b07640 1 -- 172.21.15.170:0/1763447761 >> v1:172.21.15.170:6789/0 conn(0x7f1fa00720e0 legacy=0x7f1fa006ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:18.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.114+0000 7f1fa5304640 1 -- 172.21.15.170:0/1763447761 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2956465657 0 0) 0x7f1f98001110 con 0x7f1fa00720e0 2024-05-08T04:13:18.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.114+0000 7f1fa6b07640 1 -- 172.21.15.170:0/1763447761 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa01446a0 con 0x7f1fa0144bc0 2024-05-08T04:13:18.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.115+0000 7f1fa6b07640 1 --2- 172.21.15.170:0/1763447761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fa0144bc0 0x7f1fa013d980 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00b5740 tx=0x7f1f9c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca990114dc06edd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:18.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.115+0000 7f1fa5304640 1 -- 172.21.15.170:0/1763447761 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c039070 con 0x7f1fa0144bc0 2024-05-08T04:13:18.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.115+0000 7f1fa5304640 1 -- 172.21.15.170:0/1763447761 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f9c02cbf0 con 0x7f1fa0144bc0 2024-05-08T04:13:18.118 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.115+0000 7f1fa5304640 1 -- 172.21.15.170:0/1763447761 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c034440 con 0x7f1fa0144bc0 2024-05-08T04:13:18.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.115+0000 7f1fa856a640 1 -- 172.21.15.170:0/1763447761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fa0144bc0 msgr2=0x7f1fa013d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:18.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.116+0000 7f1fa856a640 1 --2- 172.21.15.170:0/1763447761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fa0144bc0 0x7f1fa013d980 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00b5740 tx=0x7f1f9c02be40 comp rx=0 tx=0).stop 2024-05-08T04:13:18.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.116+0000 7f1fa856a640 1 -- 172.21.15.170:0/1763447761 shutdown_connections 2024-05-08T04:13:18.122 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.116+0000 7f1fa856a640 1 --2- 172.21.15.170:0/1763447761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fa0144bc0 0x7f1fa013d980 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:18.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.116+0000 7f1fa856a640 1 --2- 172.21.15.170:0/1763447761 >> v2:172.21.15.187:3300/0 conn(0x7f1fa006f580 0x7f1fa00724c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:18.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.116+0000 7f1fa856a640 1 -- 172.21.15.170:0/1763447761 >> 172.21.15.170:0/1763447761 conn(0x7f1fa00b7990 msgr2=0x7f1fa00b7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:18.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.116+0000 7f1fa856a640 1 -- 172.21.15.170:0/1763447761 shutdown_connections 2024-05-08T04:13:18.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.116+0000 7f1fa856a640 1 -- 172.21.15.170:0/1763447761 wait complete. 2024-05-08T04:13:18.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.117+0000 7f1fa856a640 1 Processor -- start 2024-05-08T04:13:18.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.117+0000 7f1fa856a640 1 -- start start 2024-05-08T04:13:18.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.117+0000 7f1fa856a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fa006f580 0x7f1fa0132cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:18.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.117+0000 7f1fa856a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fa00720e0 0x7f1fa0146d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:18.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.117+0000 7f1fa856a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1fa0144bc0 0x7f1fa01460e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:18.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.117+0000 7f1fa856a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa013e040 con 0x7f1fa00720e0 2024-05-08T04:13:18.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.117+0000 7f1fa856a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa013dec0 con 0x7f1fa006f580 2024-05-08T04:13:18.123 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.117+0000 7f1fa856a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f1fa013e1c0 con 0x7f1fa0144bc0 2024-05-08T04:13:18.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.117+0000 7f1fa5b05640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fa00720e0 0x7f1fa0146d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:18.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.117+0000 7f1fa6b07640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1fa0144bc0 0x7f1fa01460e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:18.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.117+0000 7f1fa5b05640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fa00720e0 0x7f1fa0146d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49414/0 (socket says 172.21.15.170:49414) 2024-05-08T04:13:18.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.117+0000 7f1fa5b05640 1 -- 172.21.15.170:0/2867487016 learned_addr learned my addr 172.21.15.170:0/2867487016 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:13:18.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.117+0000 7f1fa6306640 1 --2- 172.21.15.170:0/2867487016 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fa006f580 0x7f1fa0132cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:18.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.118+0000 7f1fa6b07640 1 -- 172.21.15.170:0/2867487016 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fa006f580 msgr2=0x7f1fa0132cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:18.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.118+0000 7f1fa6b07640 1 --2- 172.21.15.170:0/2867487016 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1fa006f580 0x7f1fa0132cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:18.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.118+0000 7f1fa6b07640 1 -- 172.21.15.170:0/2867487016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fa00720e0 msgr2=0x7f1fa0146d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:18.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.118+0000 7f1fa6b07640 1 --2- 172.21.15.170:0/2867487016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fa00720e0 0x7f1fa0146d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:18.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.118+0000 7f1fa6b07640 1 -- 172.21.15.170:0/2867487016 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa0148150 con 0x7f1fa0144bc0 2024-05-08T04:13:18.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.118+0000 7f1fa5b05640 1 --2- 172.21.15.170:0/2867487016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fa00720e0 0x7f1fa0146d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:13:18.124 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.118+0000 7f1fa6b07640 1 --2- 172.21.15.170:0/2867487016 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1fa0144bc0 0x7f1fa01460e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0073570 tx=0x7f1f9c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:18.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.118+0000 7f1f96ffd640 1 -- 172.21.15.170:0/2867487016 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c039050 con 0x7f1fa0144bc0 2024-05-08T04:13:18.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.118+0000 7f1f96ffd640 1 -- 172.21.15.170:0/2867487016 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f9c045070 con 0x7f1fa0144bc0 2024-05-08T04:13:18.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.118+0000 7f1f96ffd640 1 -- 172.21.15.170:0/2867487016 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c03e850 con 0x7f1fa0144bc0 2024-05-08T04:13:18.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.118+0000 7f1fa856a640 1 -- 172.21.15.170:0/2867487016 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fa0146620 con 0x7f1fa0144bc0 2024-05-08T04:13:18.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.118+0000 7f1fa856a640 1 -- 172.21.15.170:0/2867487016 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fa013f540 con 0x7f1fa0144bc0 2024-05-08T04:13:18.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.119+0000 7f1fa856a640 1 -- 172.21.15.170:0/2867487016 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1fa006d400 con 0x7f1fa0144bc0 2024-05-08T04:13:18.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.119+0000 7f1f96ffd640 1 -- 172.21.15.170:0/2867487016 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f1f9c03e9f0 con 0x7f1fa0144bc0 2024-05-08T04:13:18.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.120+0000 7f1f96ffd640 1 --2- 172.21.15.170:0/2867487016 >> v2:172.21.15.187:6816/101007897 conn(0x7f1f84076290 0x7f1f84078750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:18.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.120+0000 7f1f96ffd640 1 -- 172.21.15.170:0/2867487016 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 4525+0+0 (secure 0 0 0) 0x7f1f9c0b9ca0 con 0x7f1fa0144bc0 2024-05-08T04:13:18.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.120+0000 7f1f96ffd640 1 --2- 172.21.15.170:0/2867487016 >> v2:172.21.15.170:6804/3333432230 conn(0x7f1f8407cff0 0x7f1f8407f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:18.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.120+0000 7f1f96ffd640 1 -- 172.21.15.170:0/2867487016 --> v2:172.21.15.170:6804/3333432230 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1f8407fa80 con 0x7f1f8407cff0 2024-05-08T04:13:18.125 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.120+0000 7f1f96ffd640 1 -- 172.21.15.170:0/2867487016 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f1f9c0bc3a0 con 0x7f1fa0144bc0 2024-05-08T04:13:18.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.120+0000 7f1fa6306640 1 --2- 172.21.15.170:0/2867487016 >> v2:172.21.15.187:6816/101007897 conn(0x7f1f84076290 0x7f1f84078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:18.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.120+0000 7f1fa5b05640 1 --2- 172.21.15.170:0/2867487016 >> v2:172.21.15.170:6804/3333432230 conn(0x7f1f8407cff0 0x7f1f8407f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:18.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.120+0000 7f1fa5b05640 1 --2- 172.21.15.170:0/2867487016 >> v2:172.21.15.170:6804/3333432230 conn(0x7f1f8407cff0 0x7f1f8407f410 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:18.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.121+0000 7f1fa6306640 1 --2- 172.21.15.170:0/2867487016 >> v2:172.21.15.187:6816/101007897 conn(0x7f1f84076290 0x7f1f84078750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1f98000f60 tx=0x7f1f98007b10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:18.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.121+0000 7f1f96ffd640 1 -- 172.21.15.170:0/2867487016 <== osd.2 v2:172.21.15.170:6804/3333432230 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1f8407fa80 con 0x7f1f8407cff0 2024-05-08T04:13:18.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.134+0000 7f1fa856a640 1 -- 172.21.15.170:0/2867487016 --> v2:172.21.15.170:6804/3333432230 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f1fa013fa80 con 0x7f1f8407cff0 2024-05-08T04:13:18.136 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.134+0000 7f1f96ffd640 1 -- 172.21.15.170:0/2867487016 <== osd.2 v2:172.21.15.170:6804/3333432230 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f1fa013fa80 con 0x7f1f8407cff0 2024-05-08T04:13:18.136 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-05-08T04:13:18.136 INFO:teuthology.orchestra.run.smithi170.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T04:13:18.136 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-05-08T04:13:18.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.134+0000 7f1fa856a640 1 -- 172.21.15.170:0/2867487016 >> v2:172.21.15.170:6804/3333432230 conn(0x7f1f8407cff0 msgr2=0x7f1f8407f410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:18.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.134+0000 7f1fa856a640 1 --2- 172.21.15.170:0/2867487016 >> v2:172.21.15.170:6804/3333432230 conn(0x7f1f8407cff0 0x7f1f8407f410 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:18.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.135+0000 7f1fa856a640 1 -- 172.21.15.170:0/2867487016 >> v2:172.21.15.187:6816/101007897 conn(0x7f1f84076290 msgr2=0x7f1f84078750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:18.137 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.135+0000 7f1fa856a640 1 --2- 172.21.15.170:0/2867487016 >> v2:172.21.15.187:6816/101007897 conn(0x7f1f84076290 0x7f1f84078750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1f98000f60 tx=0x7f1f98007b10 comp rx=0 tx=0).stop 2024-05-08T04:13:18.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.135+0000 7f1fa856a640 1 -- 172.21.15.170:0/2867487016 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1fa0144bc0 msgr2=0x7f1fa01460e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:18.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.135+0000 7f1fa856a640 1 --2- 172.21.15.170:0/2867487016 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f1fa0144bc0 0x7f1fa01460e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0073570 tx=0x7f1f9c004a40 comp rx=0 tx=0).stop 2024-05-08T04:13:18.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.135+0000 7f1fa6b07640 1 -- 172.21.15.170:0/2867487016 reap_dead start 2024-05-08T04:13:18.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.135+0000 7f1fa856a640 1 -- 172.21.15.170:0/2867487016 shutdown_connections 2024-05-08T04:13:18.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.135+0000 7f1fa856a640 1 -- 172.21.15.170:0/2867487016 >> 172.21.15.170:0/2867487016 conn(0x7f1fa00b7990 msgr2=0x7f1fa00b6e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:18.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.135+0000 7f1fa856a640 1 -- 172.21.15.170:0/2867487016 shutdown_connections 2024-05-08T04:13:18.138 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.135+0000 7f1fa856a640 1 -- 172.21.15.170:0/2867487016 wait complete. 2024-05-08T04:13:18.147 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-08T04:13:18.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.245+0000 7fda8759d640 1 Processor -- start 2024-05-08T04:13:18.247 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.245+0000 7fda8759d640 1 -- start start 2024-05-08T04:13:18.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.246+0000 7fda8759d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fda801475d0 0x7fda80143190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:18.248 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.246+0000 7fda8759d640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fda801480a0 0x7fda801436d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:18.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.246+0000 7fda8759d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fda801479b0 0x7fda8006fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:18.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.246+0000 7fda8759d640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7fda80074330 con 0x7fda801480a0 2024-05-08T04:13:18.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.246+0000 7fda8759d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fda80074610 con 0x7fda801475d0 2024-05-08T04:13:18.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.246+0000 7fda8759d640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fda800748f0 con 0x7fda801479b0 2024-05-08T04:13:18.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.246+0000 7fda85b3a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fda801479b0 0x7fda8006fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:18.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.246+0000 7fda84b38640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fda801480a0 0x7fda801436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:18.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.246+0000 7fda84b38640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fda801480a0 0x7fda801436d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49420/0 (socket says 172.21.15.170:49420) 2024-05-08T04:13:18.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.246+0000 7fda84b38640 1 -- 172.21.15.170:0/3703365189 learned_addr learned my addr 172.21.15.170:0/3703365189 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:13:18.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.246+0000 7fda85b3a640 1 --2- 172.21.15.170:0/3703365189 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fda801479b0 0x7fda8006fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:48052/0 (socket says 172.21.15.170:48052) 2024-05-08T04:13:18.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.246+0000 7fda85339640 1 --2- 172.21.15.170:0/3703365189 >> v2:172.21.15.187:3300/0 conn(0x7fda801475d0 0x7fda80143190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:18.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.246+0000 7fda84b38640 1 -- 172.21.15.170:0/3703365189 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fda801479b0 msgr2=0x7fda8006fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:13:18.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.246+0000 7fda84b38640 1 --2- 172.21.15.170:0/3703365189 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fda801479b0 0x7fda8006fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:18.249 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.246+0000 7fda84b38640 1 -- 172.21.15.170:0/3703365189 >> v2:172.21.15.187:3300/0 conn(0x7fda801475d0 msgr2=0x7fda80143190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:13:18.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.246+0000 7fda84b38640 1 --2- 172.21.15.170:0/3703365189 >> v2:172.21.15.187:3300/0 conn(0x7fda801475d0 0x7fda80143190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:18.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.246+0000 7fda84b38640 1 -- 172.21.15.170:0/3703365189 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda800ae8c0 con 0x7fda801480a0 2024-05-08T04:13:18.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.246+0000 7fda85b3a640 1 --2- 172.21.15.170:0/3703365189 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fda801479b0 0x7fda8006fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:13:18.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.246+0000 7fda85339640 1 --2- 172.21.15.170:0/3703365189 >> v2:172.21.15.187:3300/0 conn(0x7fda801475d0 0x7fda80143190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T04:13:18.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.247+0000 7fda84b38640 1 --2- 172.21.15.170:0/3703365189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda801480a0 0x7fda801436d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fda80074820 tx=0x7fda7402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f477537bbaf44e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:18.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.247+0000 7fda7ffff640 1 -- 172.21.15.170:0/3703365189 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda740317c0 con 0x7fda801480a0 2024-05-08T04:13:18.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.247+0000 7fda7ffff640 1 -- 172.21.15.170:0/3703365189 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda74037070 con 0x7fda801480a0 2024-05-08T04:13:18.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.248+0000 7fda7ffff640 1 -- 172.21.15.170:0/3703365189 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74031b80 con 0x7fda801480a0 2024-05-08T04:13:18.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.248+0000 7fda8759d640 1 -- 172.21.15.170:0/3703365189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda801480a0 msgr2=0x7fda801436d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:18.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.248+0000 7fda8759d640 1 --2- 172.21.15.170:0/3703365189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda801480a0 0x7fda801436d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fda80074820 tx=0x7fda7402b860 comp rx=0 tx=0).stop 2024-05-08T04:13:18.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.248+0000 7fda8759d640 1 -- 172.21.15.170:0/3703365189 shutdown_connections 2024-05-08T04:13:18.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.248+0000 7fda8759d640 1 --2- 172.21.15.170:0/3703365189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda801480a0 0x7fda801436d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:18.251 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.248+0000 7fda8759d640 1 --2- 172.21.15.170:0/3703365189 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fda801479b0 0x7fda8006fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:18.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.248+0000 7fda8759d640 1 --2- 172.21.15.170:0/3703365189 >> v2:172.21.15.187:3300/0 conn(0x7fda801475d0 0x7fda80143190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:18.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.248+0000 7fda8759d640 1 -- 172.21.15.170:0/3703365189 >> 172.21.15.170:0/3703365189 conn(0x7fda800b9e60 msgr2=0x7fda800ba270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:18.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.248+0000 7fda8759d640 1 -- 172.21.15.170:0/3703365189 shutdown_connections 2024-05-08T04:13:18.252 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.248+0000 7fda8759d640 1 -- 172.21.15.170:0/3703365189 wait complete. 2024-05-08T04:13:18.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.250+0000 7fda8759d640 1 Processor -- start 2024-05-08T04:13:18.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.250+0000 7fda8759d640 1 -- start start 2024-05-08T04:13:18.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.251+0000 7fda8759d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fda801475d0 0x7fda801ce240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:18.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.251+0000 7fda8759d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda801479b0 0x7fda801dcb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:18.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.251+0000 7fda8759d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda801d0430 0x7fda801dd040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:18.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.251+0000 7fda8759d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fda800705b0 con 0x7fda801479b0 2024-05-08T04:13:18.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.251+0000 7fda8759d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fda80070430 con 0x7fda801d0430 2024-05-08T04:13:18.254 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.251+0000 7fda8759d640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fda80070730 con 0x7fda801475d0 2024-05-08T04:13:18.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.251+0000 7fda84b38640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda801479b0 0x7fda801dcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:18.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.251+0000 7fda85339640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fda801475d0 0x7fda801ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:18.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.251+0000 7fda85339640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fda801475d0 0x7fda801ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:48062/0 (socket says 172.21.15.170:48062) 2024-05-08T04:13:18.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.251+0000 7fda84b38640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda801479b0 0x7fda801dcb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49430/0 (socket says 172.21.15.170:49430) 2024-05-08T04:13:18.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.251+0000 7fda85339640 1 -- 172.21.15.170:0/3282550266 learned_addr learned my addr 172.21.15.170:0/3282550266 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:13:18.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.251+0000 7fda85b3a640 1 --2- 172.21.15.170:0/3282550266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda801d0430 0x7fda801dd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:18.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.251+0000 7fda84b38640 1 -- 172.21.15.170:0/3282550266 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fda801475d0 msgr2=0x7fda801ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:18.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.251+0000 7fda84b38640 1 --2- 172.21.15.170:0/3282550266 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fda801475d0 0x7fda801ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:18.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.251+0000 7fda84b38640 1 -- 172.21.15.170:0/3282550266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda801d0430 msgr2=0x7fda801dd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:18.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.251+0000 7fda84b38640 1 --2- 172.21.15.170:0/3282550266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda801d0430 0x7fda801dd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:18.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.251+0000 7fda84b38640 1 -- 172.21.15.170:0/3282550266 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda800afa00 con 0x7fda801479b0 2024-05-08T04:13:18.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.251+0000 7fda85339640 1 --2- 172.21.15.170:0/3282550266 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fda801475d0 0x7fda801ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:13:18.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.251+0000 7fda84b38640 1 --2- 172.21.15.170:0/3282550266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda801479b0 0x7fda801dcb00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fda74033000 tx=0x7fda740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:18.255 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.251+0000 7fda7dffb640 1 -- 172.21.15.170:0/3282550266 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74032860 con 0x7fda801479b0 2024-05-08T04:13:18.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.251+0000 7fda7dffb640 1 -- 172.21.15.170:0/3282550266 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda74032a40 con 0x7fda801479b0 2024-05-08T04:13:18.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.251+0000 7fda8759d640 1 -- 172.21.15.170:0/3282550266 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda800afce0 con 0x7fda801479b0 2024-05-08T04:13:18.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.251+0000 7fda7dffb640 1 -- 172.21.15.170:0/3282550266 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74032e30 con 0x7fda801479b0 2024-05-08T04:13:18.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.252+0000 7fda8759d640 1 -- 172.21.15.170:0/3282550266 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda801dd580 con 0x7fda801479b0 2024-05-08T04:13:18.256 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.253+0000 7fda8759d640 1 -- 172.21.15.170:0/3282550266 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda8006e9e0 con 0x7fda801479b0 2024-05-08T04:13:18.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.253+0000 7fda7dffb640 1 -- 172.21.15.170:0/3282550266 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7fda7404c060 con 0x7fda801479b0 2024-05-08T04:13:18.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.254+0000 7fda7dffb640 1 --2- 172.21.15.170:0/3282550266 >> v2:172.21.15.187:6816/101007897 conn(0x7fda6c076290 0x7fda6c078750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:18.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.254+0000 7fda7dffb640 1 -- 172.21.15.170:0/3282550266 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 4525+0+0 (secure 0 0 0) 0x7fda740b89f0 con 0x7fda801479b0 2024-05-08T04:13:18.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.254+0000 7fda85339640 1 --2- 172.21.15.170:0/3282550266 >> v2:172.21.15.187:6816/101007897 conn(0x7fda6c076290 0x7fda6c078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:18.257 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.254+0000 7fda85339640 1 --2- 172.21.15.170:0/3282550266 >> v2:172.21.15.187:6816/101007897 conn(0x7fda6c076290 0x7fda6c078750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fda70003c20 tx=0x7fda700099d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:18.260 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.258+0000 7fda7dffb640 1 -- 172.21.15.170:0/3282550266 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fda74040070 con 0x7fda801479b0 2024-05-08T04:13:18.441 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.439+0000 7fda8759d640 1 -- 172.21.15.170:0/3282550266 --> v2:172.21.15.187:6816/101007897 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fda800b97e0 con 0x7fda6c076290 2024-05-08T04:13:18.442 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.440+0000 7fda7dffb640 1 -- 172.21.15.170:0/3282550266 <== mgr.4107 v2:172.21.15.187:6816/101007897 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fda800b97e0 con 0x7fda6c076290 2024-05-08T04:13:18.442 INFO:teuthology.orchestra.run.smithi170.stderr:instructed osd(s) 2 to deep-scrub 2024-05-08T04:13:18.444 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.442+0000 7fda8759d640 1 -- 172.21.15.170:0/3282550266 >> v2:172.21.15.187:6816/101007897 conn(0x7fda6c076290 msgr2=0x7fda6c078750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:18.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.442+0000 7fda8759d640 1 --2- 172.21.15.170:0/3282550266 >> v2:172.21.15.187:6816/101007897 conn(0x7fda6c076290 0x7fda6c078750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fda70003c20 tx=0x7fda700099d0 comp rx=0 tx=0).stop 2024-05-08T04:13:18.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.442+0000 7fda8759d640 1 -- 172.21.15.170:0/3282550266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda801479b0 msgr2=0x7fda801dcb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:18.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.442+0000 7fda8759d640 1 --2- 172.21.15.170:0/3282550266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda801479b0 0x7fda801dcb00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fda74033000 tx=0x7fda740046c0 comp rx=0 tx=0).stop 2024-05-08T04:13:18.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.442+0000 7fda8759d640 1 -- 172.21.15.170:0/3282550266 shutdown_connections 2024-05-08T04:13:18.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.442+0000 7fda8759d640 1 --2- 172.21.15.170:0/3282550266 >> v2:172.21.15.187:6816/101007897 conn(0x7fda6c076290 0x7fda6c078750 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:18.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.442+0000 7fda8759d640 1 --2- 172.21.15.170:0/3282550266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fda801d0430 0x7fda801dd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:18.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.442+0000 7fda8759d640 1 --2- 172.21.15.170:0/3282550266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fda801479b0 0x7fda801dcb00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:18.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.442+0000 7fda8759d640 1 --2- 172.21.15.170:0/3282550266 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fda801475d0 0x7fda801ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:18.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.442+0000 7fda8759d640 1 -- 172.21.15.170:0/3282550266 >> 172.21.15.170:0/3282550266 conn(0x7fda800b9e60 msgr2=0x7fda8013fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:18.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.442+0000 7fda8759d640 1 -- 172.21.15.170:0/3282550266 shutdown_connections 2024-05-08T04:13:18.445 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.443+0000 7fda8759d640 1 -- 172.21.15.170:0/3282550266 wait complete. 2024-05-08T04:13:18.455 INFO:tasks.ceph:Scrubbing osd.3 2024-05-08T04:13:18.455 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-08T04:13:18.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.557+0000 7f642f76e640 1 Processor -- start 2024-05-08T04:13:18.559 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.557+0000 7f642f76e640 1 -- start start 2024-05-08T04:13:18.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.558+0000 7f642f76e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6428147e20 0x7f6428070b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:18.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.558+0000 7f642f76e640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6428073ad0 con 0x7f6428147520 2024-05-08T04:13:18.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.558+0000 7f642f76e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6428147900 con 0x7f64280710b0 2024-05-08T04:13:18.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.558+0000 7f642f76e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f6428073d90 con 0x7f6428147e20 2024-05-08T04:13:18.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.558+0000 7f642cd09640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6428147e20 0x7f6428070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:18.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.558+0000 7f642d50a640 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f6428147520 0x7f6428143260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43766/0 (socket says 172.21.15.170:43766) 2024-05-08T04:13:18.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.558+0000 7f642d50a640 1 -- 172.21.15.170:0/2011789402 learned_addr learned my addr 172.21.15.170:0/2011789402 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-05-08T04:13:18.560 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.558+0000 7f641ffff640 1 -- 172.21.15.170:0/2011789402 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 109508661 0 0) 0x7f6428073ad0 con 0x7f6428147520 2024-05-08T04:13:18.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.558+0000 7f642cd09640 1 -- 172.21.15.170:0/2011789402 >> v1:172.21.15.187:6789/0 conn(0x7f64280710b0 legacy=0x7f642814e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:18.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.558+0000 7f642cd09640 1 -- 172.21.15.170:0/2011789402 >> v1:172.21.15.170:6789/0 conn(0x7f6428147520 legacy=0x7f6428143260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:18.561 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.558+0000 7f642cd09640 1 -- 172.21.15.170:0/2011789402 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6428072700 con 0x7f6428147e20 2024-05-08T04:13:18.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.559+0000 7f642cd09640 1 --2- 172.21.15.170:0/2011789402 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6428147e20 0x7f6428070b70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f64240097d0 tx=0x7f642402b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9f931c7e6851b7a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:18.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.559+0000 7f641ffff640 1 -- 172.21.15.170:0/2011789402 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6424035020 con 0x7f6428147e20 2024-05-08T04:13:18.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.559+0000 7f641ffff640 1 -- 172.21.15.170:0/2011789402 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6424031890 con 0x7f6428147e20 2024-05-08T04:13:18.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.559+0000 7f641ffff640 1 -- 172.21.15.170:0/2011789402 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6424031c60 con 0x7f6428147e20 2024-05-08T04:13:18.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.560+0000 7f642f76e640 1 -- 172.21.15.170:0/2011789402 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6428147e20 msgr2=0x7f6428070b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:18.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.560+0000 7f642f76e640 1 --2- 172.21.15.170:0/2011789402 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6428147e20 0x7f6428070b70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f64240097d0 tx=0x7f642402b930 comp rx=0 tx=0).stop 2024-05-08T04:13:18.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.560+0000 7f642f76e640 1 -- 172.21.15.170:0/2011789402 shutdown_connections 2024-05-08T04:13:18.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.560+0000 7f642f76e640 1 --2- 172.21.15.170:0/2011789402 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6428147e20 0x7f6428070b70 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:18.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.560+0000 7f642f76e640 1 -- 172.21.15.170:0/2011789402 >> 172.21.15.170:0/2011789402 conn(0x7f64280b64d0 msgr2=0x7f64280b68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:18.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.560+0000 7f642f76e640 1 -- 172.21.15.170:0/2011789402 shutdown_connections 2024-05-08T04:13:18.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.560+0000 7f642f76e640 1 -- 172.21.15.170:0/2011789402 wait complete. 2024-05-08T04:13:18.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.561+0000 7f642f76e640 1 Processor -- start 2024-05-08T04:13:18.563 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.561+0000 7f642f76e640 1 -- start start 2024-05-08T04:13:18.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.561+0000 7f642f76e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f64280710b0 0x7f6428142370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:18.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.561+0000 7f642f76e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6428147520 0x7f642813e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:18.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.561+0000 7f642f76e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6428147e20 0x7f642813bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:18.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.561+0000 7f642f76e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6428143af0 con 0x7f6428147520 2024-05-08T04:13:18.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.561+0000 7f642f76e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6428143970 con 0x7f6428147e20 2024-05-08T04:13:18.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.561+0000 7f642f76e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f642814fd60 con 0x7f64280710b0 2024-05-08T04:13:18.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.561+0000 7f642d50a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f64280710b0 0x7f6428142370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:18.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.561+0000 7f642cd09640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6428147520 0x7f642813e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:18.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.561+0000 7f642cd09640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6428147520 0x7f642813e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49446/0 (socket says 172.21.15.170:49446) 2024-05-08T04:13:18.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.561+0000 7f642cd09640 1 -- 172.21.15.170:0/3716671433 learned_addr learned my addr 172.21.15.170:0/3716671433 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:13:18.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.561+0000 7f642d50a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f64280710b0 0x7f6428142370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:48070/0 (socket says 172.21.15.170:48070) 2024-05-08T04:13:18.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.562+0000 7f642dd0b640 1 --2- 172.21.15.170:0/3716671433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6428147e20 0x7f642813bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:18.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.562+0000 7f642d50a640 1 -- 172.21.15.170:0/3716671433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6428147e20 msgr2=0x7f642813bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:18.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.562+0000 7f642d50a640 1 --2- 172.21.15.170:0/3716671433 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6428147e20 0x7f642813bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:18.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.562+0000 7f642d50a640 1 -- 172.21.15.170:0/3716671433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6428147520 msgr2=0x7f642813e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:18.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.562+0000 7f642d50a640 1 --2- 172.21.15.170:0/3716671433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6428147520 0x7f642813e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:18.567 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.562+0000 7f642d50a640 1 -- 172.21.15.170:0/3716671433 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f642813db60 con 0x7f64280710b0 2024-05-08T04:13:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.562+0000 7f642cd09640 1 --2- 172.21.15.170:0/3716671433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6428147520 0x7f642813e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:13:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.562+0000 7f642d50a640 1 --2- 172.21.15.170:0/3716671433 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f64280710b0 0x7f6428142370 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f641800c3e0 tx=0x7f641800c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.562+0000 7f641dffb640 1 -- 172.21.15.170:0/3716671433 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64180074e0 con 0x7f64280710b0 2024-05-08T04:13:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.562+0000 7f641dffb640 1 -- 172.21.15.170:0/3716671433 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64180047e0 con 0x7f64280710b0 2024-05-08T04:13:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.562+0000 7f642f76e640 1 -- 172.21.15.170:0/3716671433 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f642813c510 con 0x7f64280710b0 2024-05-08T04:13:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.562+0000 7f641dffb640 1 -- 172.21.15.170:0/3716671433 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6418004bd0 con 0x7f64280710b0 2024-05-08T04:13:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.562+0000 7f642f76e640 1 -- 172.21.15.170:0/3716671433 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f642813e870 con 0x7f64280710b0 2024-05-08T04:13:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.563+0000 7f642f76e640 1 -- 172.21.15.170:0/3716671433 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f642806e9e0 con 0x7f64280710b0 2024-05-08T04:13:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.564+0000 7f641dffb640 1 -- 172.21.15.170:0/3716671433 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f6418011070 con 0x7f64280710b0 2024-05-08T04:13:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.564+0000 7f641dffb640 1 --2- 172.21.15.170:0/3716671433 >> v2:172.21.15.187:6816/101007897 conn(0x7f640c076290 0x7f640c078750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.564+0000 7f641dffb640 1 -- 172.21.15.170:0/3716671433 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 4525+0+0 (secure 0 0 0) 0x7f6418096be0 con 0x7f64280710b0 2024-05-08T04:13:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.564+0000 7f641dffb640 1 --2- 172.21.15.170:0/3716671433 >> v2:172.21.15.170:6812/153555120 conn(0x7f640c07cff0 0x7f640c07f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.564+0000 7f641dffb640 1 -- 172.21.15.170:0/3716671433 --> v2:172.21.15.170:6812/153555120 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f640c07fa80 con 0x7f640c07cff0 2024-05-08T04:13:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.564+0000 7f641dffb640 1 -- 172.21.15.170:0/3716671433 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f64180992e0 con 0x7f64280710b0 2024-05-08T04:13:18.568 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.564+0000 7f642cd09640 1 --2- 172.21.15.170:0/3716671433 >> v2:172.21.15.187:6816/101007897 conn(0x7f640c076290 0x7f640c078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:18.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.564+0000 7f642dd0b640 1 --2- 172.21.15.170:0/3716671433 >> v2:172.21.15.170:6812/153555120 conn(0x7f640c07cff0 0x7f640c07f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:18.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.565+0000 7f642dd0b640 1 --2- 172.21.15.170:0/3716671433 >> v2:172.21.15.170:6812/153555120 conn(0x7f640c07cff0 0x7f640c07f410 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:18.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.565+0000 7f642cd09640 1 --2- 172.21.15.170:0/3716671433 >> v2:172.21.15.187:6816/101007897 conn(0x7f640c076290 0x7f640c078750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6424000c00 tx=0x7f6424008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:18.569 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.565+0000 7f641dffb640 1 -- 172.21.15.170:0/3716671433 <== osd.3 v2:172.21.15.170:6812/153555120 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f640c07fa80 con 0x7f640c07cff0 2024-05-08T04:13:18.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.578+0000 7f642f76e640 1 -- 172.21.15.170:0/3716671433 --> v2:172.21.15.170:6812/153555120 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f642813edb0 con 0x7f640c07cff0 2024-05-08T04:13:18.580 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.578+0000 7f641dffb640 1 -- 172.21.15.170:0/3716671433 <== osd.3 v2:172.21.15.170:6812/153555120 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f642813edb0 con 0x7f640c07cff0 2024-05-08T04:13:18.580 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-05-08T04:13:18.580 INFO:teuthology.orchestra.run.smithi170.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T04:13:18.580 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-05-08T04:13:18.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.579+0000 7f642f76e640 1 -- 172.21.15.170:0/3716671433 >> v2:172.21.15.170:6812/153555120 conn(0x7f640c07cff0 msgr2=0x7f640c07f410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:18.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.579+0000 7f642f76e640 1 --2- 172.21.15.170:0/3716671433 >> v2:172.21.15.170:6812/153555120 conn(0x7f640c07cff0 0x7f640c07f410 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:18.581 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.579+0000 7f642f76e640 1 -- 172.21.15.170:0/3716671433 >> v2:172.21.15.187:6816/101007897 conn(0x7f640c076290 msgr2=0x7f640c078750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:18.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.579+0000 7f642f76e640 1 --2- 172.21.15.170:0/3716671433 >> v2:172.21.15.187:6816/101007897 conn(0x7f640c076290 0x7f640c078750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6424000c00 tx=0x7f6424008030 comp rx=0 tx=0).stop 2024-05-08T04:13:18.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.579+0000 7f642f76e640 1 -- 172.21.15.170:0/3716671433 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f64280710b0 msgr2=0x7f6428142370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:18.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.579+0000 7f642f76e640 1 --2- 172.21.15.170:0/3716671433 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f64280710b0 0x7f6428142370 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f641800c3e0 tx=0x7f641800c8a0 comp rx=0 tx=0).stop 2024-05-08T04:13:18.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.579+0000 7f642dd0b640 1 -- 172.21.15.170:0/3716671433 reap_dead start 2024-05-08T04:13:18.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.579+0000 7f642f76e640 1 -- 172.21.15.170:0/3716671433 shutdown_connections 2024-05-08T04:13:18.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.579+0000 7f642f76e640 1 -- 172.21.15.170:0/3716671433 >> 172.21.15.170:0/3716671433 conn(0x7f64280b64d0 msgr2=0x7f64280b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:18.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.579+0000 7f642f76e640 1 -- 172.21.15.170:0/3716671433 shutdown_connections 2024-05-08T04:13:18.582 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.579+0000 7f642f76e640 1 -- 172.21.15.170:0/3716671433 wait complete. 2024-05-08T04:13:18.591 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-08T04:13:18.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.689+0000 7f125882a640 1 Processor -- start 2024-05-08T04:13:18.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.689+0000 7f125882a640 1 -- start start 2024-05-08T04:13:18.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.690+0000 7f125882a640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f12540b4e10 0x7f12540aae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:18.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.690+0000 7f125882a640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12540b51f0 con 0x7f12540738e0 2024-05-08T04:13:18.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.690+0000 7f125882a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12540ac780 con 0x7f1254072d60 2024-05-08T04:13:18.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.690+0000 7f125882a640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7f12540aca60 con 0x7f12540b4e10 2024-05-08T04:13:18.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.690+0000 7f125259c640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f12540b4e10 0x7f12540aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:18.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.690+0000 7f1252d9d640 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f12540738e0 0x7f125413b2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43768/0 (socket says 172.21.15.170:43768) 2024-05-08T04:13:18.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.690+0000 7f1252d9d640 1 -- 172.21.15.170:0/3135104901 learned_addr learned my addr 172.21.15.170:0/3135104901 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-05-08T04:13:18.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.690+0000 7f125259c640 1 -- 172.21.15.170:0/3135104901 >> v1:172.21.15.187:6789/0 conn(0x7f1254072d60 legacy=0x7f12540731d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:13:18.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.690+0000 7f125259c640 1 -- 172.21.15.170:0/3135104901 >> v1:172.21.15.170:6789/0 conn(0x7f12540738e0 legacy=0x7f125413b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:18.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.690+0000 7f1251d9b640 1 -- 172.21.15.170:0/3135104901 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 456868029 0 0) 0x7f12540b51f0 con 0x7f12540738e0 2024-05-08T04:13:18.693 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.690+0000 7f125259c640 1 -- 172.21.15.170:0/3135104901 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12540ab440 con 0x7f12540b4e10 2024-05-08T04:13:18.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.691+0000 7f125259c640 1 --2- 172.21.15.170:0/3135104901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12540b4e10 0x7f12540aae40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f12440097a0 tx=0x7f124402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9289ae23cfeb2443 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:18.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.691+0000 7f1251d9b640 1 -- 172.21.15.170:0/3135104901 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12440315a0 con 0x7f12540b4e10 2024-05-08T04:13:18.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.691+0000 7f1251d9b640 1 -- 172.21.15.170:0/3135104901 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1244037070 con 0x7f12540b4e10 2024-05-08T04:13:18.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.691+0000 7f1251d9b640 1 -- 172.21.15.170:0/3135104901 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1244031960 con 0x7f12540b4e10 2024-05-08T04:13:18.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.692+0000 7f125882a640 1 -- 172.21.15.170:0/3135104901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12540b4e10 msgr2=0x7f12540aae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:18.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.692+0000 7f125882a640 1 --2- 172.21.15.170:0/3135104901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12540b4e10 0x7f12540aae40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f12440097a0 tx=0x7f124402b840 comp rx=0 tx=0).stop 2024-05-08T04:13:18.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.692+0000 7f125882a640 1 -- 172.21.15.170:0/3135104901 shutdown_connections 2024-05-08T04:13:18.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.692+0000 7f125882a640 1 --2- 172.21.15.170:0/3135104901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12540b4e10 0x7f12540aae40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:18.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.692+0000 7f125882a640 1 -- 172.21.15.170:0/3135104901 >> 172.21.15.170:0/3135104901 conn(0x7f12540b7fc0 msgr2=0x7f12540b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:18.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.692+0000 7f125882a640 1 -- 172.21.15.170:0/3135104901 shutdown_connections 2024-05-08T04:13:18.695 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.692+0000 7f125882a640 1 -- 172.21.15.170:0/3135104901 wait complete. 2024-05-08T04:13:18.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.693+0000 7f125882a640 1 Processor -- start 2024-05-08T04:13:18.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.693+0000 7f125882a640 1 -- start start 2024-05-08T04:13:18.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.694+0000 7f125882a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1254072d60 0x7f125413f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:18.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.694+0000 7f125882a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12540738e0 0x7f12541d7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:18.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.694+0000 7f125882a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f12540b4e10 0x7f1254141bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:18.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.694+0000 7f125882a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f125413bd60 con 0x7f12540738e0 2024-05-08T04:13:18.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.694+0000 7f125882a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f125413bbe0 con 0x7f1254072d60 2024-05-08T04:13:18.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.694+0000 7f125882a640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f125413bee0 con 0x7f12540b4e10 2024-05-08T04:13:18.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.694+0000 7f1252d9d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f12540b4e10 0x7f1254141bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:18.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.694+0000 7f1249d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12540738e0 0x7f12541d7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:18.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.694+0000 7f1249d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12540738e0 0x7f12541d7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49474/0 (socket says 172.21.15.170:49474) 2024-05-08T04:13:18.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.694+0000 7f1252d9d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f12540b4e10 0x7f1254141bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:48080/0 (socket says 172.21.15.170:48080) 2024-05-08T04:13:18.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.694+0000 7f1249d9b640 1 -- 172.21.15.170:0/3531281222 learned_addr learned my addr 172.21.15.170:0/3531281222 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:13:18.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.694+0000 7f125259c640 1 --2- 172.21.15.170:0/3531281222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1254072d60 0x7f125413f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:18.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.694+0000 7f1252d9d640 1 -- 172.21.15.170:0/3531281222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1254072d60 msgr2=0x7f125413f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:18.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.694+0000 7f1252d9d640 1 --2- 172.21.15.170:0/3531281222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1254072d60 0x7f125413f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:18.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.694+0000 7f1252d9d640 1 -- 172.21.15.170:0/3531281222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12540738e0 msgr2=0x7f12541d7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:18.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.694+0000 7f1252d9d640 1 --2- 172.21.15.170:0/3531281222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12540738e0 0x7f12541d7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:18.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.694+0000 7f1252d9d640 1 -- 172.21.15.170:0/3531281222 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12540ae690 con 0x7f12540b4e10 2024-05-08T04:13:18.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.694+0000 7f1249d9b640 1 --2- 172.21.15.170:0/3531281222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12540738e0 0x7f12541d7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:13:18.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.694+0000 7f1252d9d640 1 --2- 172.21.15.170:0/3531281222 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f12540b4e10 0x7f1254141bc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f124c000f80 tx=0x7f124c00cb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:18.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.695+0000 7f124b7fe640 1 -- 172.21.15.170:0/3531281222 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c018070 con 0x7f12540b4e10 2024-05-08T04:13:18.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.695+0000 7f124b7fe640 1 -- 172.21.15.170:0/3531281222 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f124c004d80 con 0x7f12540b4e10 2024-05-08T04:13:18.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.695+0000 7f124b7fe640 1 -- 172.21.15.170:0/3531281222 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c00a880 con 0x7f12540b4e10 2024-05-08T04:13:18.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.695+0000 7f125882a640 1 -- 172.21.15.170:0/3531281222 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12540ae930 con 0x7f12540b4e10 2024-05-08T04:13:18.699 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.695+0000 7f125882a640 1 -- 172.21.15.170:0/3531281222 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f12541d7ac0 con 0x7f12540b4e10 2024-05-08T04:13:18.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.696+0000 7f125882a640 1 -- 172.21.15.170:0/3531281222 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f125406e9e0 con 0x7f12540b4e10 2024-05-08T04:13:18.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.696+0000 7f124b7fe640 1 -- 172.21.15.170:0/3531281222 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f124c00ac30 con 0x7f12540b4e10 2024-05-08T04:13:18.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.697+0000 7f124b7fe640 1 --2- 172.21.15.170:0/3531281222 >> v2:172.21.15.187:6816/101007897 conn(0x7f1234076290 0x7f1234078750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:18.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.697+0000 7f124b7fe640 1 -- 172.21.15.170:0/3531281222 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 4525+0+0 (secure 0 0 0) 0x7f124c096810 con 0x7f12540b4e10 2024-05-08T04:13:18.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.697+0000 7f125259c640 1 --2- 172.21.15.170:0/3531281222 >> v2:172.21.15.187:6816/101007897 conn(0x7f1234076290 0x7f1234078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:18.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.697+0000 7f125259c640 1 --2- 172.21.15.170:0/3531281222 >> v2:172.21.15.187:6816/101007897 conn(0x7f1234076290 0x7f1234078750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1244036000 tx=0x7f124403d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:18.703 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.701+0000 7f124b7fe640 1 -- 172.21.15.170:0/3531281222 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f124c05ffc0 con 0x7f12540b4e10 2024-05-08T04:13:18.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.882+0000 7f125882a640 1 -- 172.21.15.170:0/3531281222 --> v2:172.21.15.187:6816/101007897 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f12541d6fc0 con 0x7f1234076290 2024-05-08T04:13:18.884 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.882+0000 7f124b7fe640 1 -- 172.21.15.170:0/3531281222 <== mgr.4107 v2:172.21.15.187:6816/101007897 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f12541d6fc0 con 0x7f1234076290 2024-05-08T04:13:18.884 INFO:teuthology.orchestra.run.smithi170.stderr:instructed osd(s) 3 to deep-scrub 2024-05-08T04:13:18.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.885+0000 7f125882a640 1 -- 172.21.15.170:0/3531281222 >> v2:172.21.15.187:6816/101007897 conn(0x7f1234076290 msgr2=0x7f1234078750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:18.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.885+0000 7f125882a640 1 --2- 172.21.15.170:0/3531281222 >> v2:172.21.15.187:6816/101007897 conn(0x7f1234076290 0x7f1234078750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1244036000 tx=0x7f124403d040 comp rx=0 tx=0).stop 2024-05-08T04:13:18.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.885+0000 7f125882a640 1 -- 172.21.15.170:0/3531281222 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f12540b4e10 msgr2=0x7f1254141bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:18.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.885+0000 7f125882a640 1 --2- 172.21.15.170:0/3531281222 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f12540b4e10 0x7f1254141bc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f124c000f80 tx=0x7f124c00cb90 comp rx=0 tx=0).stop 2024-05-08T04:13:18.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.885+0000 7f125882a640 1 -- 172.21.15.170:0/3531281222 shutdown_connections 2024-05-08T04:13:18.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.885+0000 7f125882a640 1 --2- 172.21.15.170:0/3531281222 >> v2:172.21.15.187:6816/101007897 conn(0x7f1234076290 0x7f1234078750 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:18.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.885+0000 7f125882a640 1 --2- 172.21.15.170:0/3531281222 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f12540b4e10 0x7f1254141bc0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:18.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.885+0000 7f125882a640 1 --2- 172.21.15.170:0/3531281222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f12540738e0 0x7f12541d7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:18.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.885+0000 7f125882a640 1 --2- 172.21.15.170:0/3531281222 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1254072d60 0x7f125413f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:18.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.885+0000 7f125882a640 1 -- 172.21.15.170:0/3531281222 >> 172.21.15.170:0/3531281222 conn(0x7f12540b7fc0 msgr2=0x7f1254075c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:18.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.885+0000 7f125882a640 1 -- 172.21.15.170:0/3531281222 shutdown_connections 2024-05-08T04:13:18.887 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:18.885+0000 7f125882a640 1 -- 172.21.15.170:0/3531281222 wait complete. 2024-05-08T04:13:18.900 INFO:tasks.ceph:Scrubbing osd.4 2024-05-08T04:13:18.900 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-08T04:13:19.002 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.001+0000 7fc644fd0640 1 Processor -- start 2024-05-08T04:13:19.003 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.001+0000 7fc644fd0640 1 -- start start 2024-05-08T04:13:19.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.002+0000 7fc644fd0640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc640147520 0x7fc640147e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.002+0000 7fc644fd0640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc640143260 0x7fc6401436a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.002+0000 7fc644fd0640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc640070b70 0x7fc640071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.002+0000 7fc644fd0640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7fc640073ad0 con 0x7fc640143260 2024-05-08T04:13:19.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.002+0000 7fc644fd0640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc640147900 con 0x7fc640147520 2024-05-08T04:13:19.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.002+0000 7fc644fd0640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fc640073d90 con 0x7fc640070b70 2024-05-08T04:13:19.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.002+0000 7fc63ed9d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc640070b70 0x7fc640071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.002+0000 7fc637fff640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc640143260 0x7fc6401436a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.002+0000 7fc63ed9d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc640070b70 0x7fc640071000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:48096/0 (socket says 172.21.15.170:48096) 2024-05-08T04:13:19.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.002+0000 7fc637fff640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc640143260 0x7fc6401436a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49480/0 (socket says 172.21.15.170:49480) 2024-05-08T04:13:19.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.002+0000 7fc63ed9d640 1 -- 172.21.15.170:0/1081646111 learned_addr learned my addr 172.21.15.170:0/1081646111 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:13:19.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.002+0000 7fc63e59c640 1 --2- 172.21.15.170:0/1081646111 >> v2:172.21.15.187:3300/0 conn(0x7fc640147520 0x7fc640147e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.002+0000 7fc63ed9d640 1 -- 172.21.15.170:0/1081646111 >> v2:172.21.15.187:3300/0 conn(0x7fc640147520 msgr2=0x7fc640147e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:13:19.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.002+0000 7fc63ed9d640 1 --2- 172.21.15.170:0/1081646111 >> v2:172.21.15.187:3300/0 conn(0x7fc640147520 0x7fc640147e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.002+0000 7fc63ed9d640 1 -- 172.21.15.170:0/1081646111 >> v2:172.21.15.170:3300/0 conn(0x7fc640143260 msgr2=0x7fc6401436a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:13:19.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.002+0000 7fc63ed9d640 1 --2- 172.21.15.170:0/1081646111 >> v2:172.21.15.170:3300/0 conn(0x7fc640143260 0x7fc6401436a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.002+0000 7fc63ed9d640 1 -- 172.21.15.170:0/1081646111 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc640072700 con 0x7fc640070b70 2024-05-08T04:13:19.005 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.002+0000 7fc637fff640 1 --2- 172.21.15.170:0/1081646111 >> v2:172.21.15.170:3300/0 conn(0x7fc640143260 0x7fc6401436a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:13:19.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.003+0000 7fc63ed9d640 1 --2- 172.21.15.170:0/1081646111 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc640070b70 0x7fc640071000 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc64006d5b0 tx=0x7fc63802bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10abc7b238d82234 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:19.008 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.003+0000 7fc63dd9b640 1 -- 172.21.15.170:0/1081646111 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63802ca70 con 0x7fc640070b70 2024-05-08T04:13:19.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.003+0000 7fc63dd9b640 1 -- 172.21.15.170:0/1081646111 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc63802cc10 con 0x7fc640070b70 2024-05-08T04:13:19.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.003+0000 7fc63dd9b640 1 -- 172.21.15.170:0/1081646111 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6380343f0 con 0x7fc640070b70 2024-05-08T04:13:19.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.003+0000 7fc644fd0640 1 -- 172.21.15.170:0/1081646111 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc640070b70 msgr2=0x7fc640071000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:19.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.003+0000 7fc644fd0640 1 --2- 172.21.15.170:0/1081646111 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc640070b70 0x7fc640071000 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc64006d5b0 tx=0x7fc63802bed0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.003+0000 7fc644fd0640 1 -- 172.21.15.170:0/1081646111 shutdown_connections 2024-05-08T04:13:19.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.003+0000 7fc644fd0640 1 --2- 172.21.15.170:0/1081646111 >> v2:172.21.15.170:3300/0 conn(0x7fc640143260 0x7fc6401436a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.003+0000 7fc644fd0640 1 --2- 172.21.15.170:0/1081646111 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc640070b70 0x7fc640071000 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.003+0000 7fc644fd0640 1 --2- 172.21.15.170:0/1081646111 >> v2:172.21.15.187:3300/0 conn(0x7fc640147520 0x7fc640147e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.004+0000 7fc644fd0640 1 -- 172.21.15.170:0/1081646111 >> 172.21.15.170:0/1081646111 conn(0x7fc6400b64d0 msgr2=0x7fc6400b68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:19.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.004+0000 7fc644fd0640 1 -- 172.21.15.170:0/1081646111 shutdown_connections 2024-05-08T04:13:19.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.004+0000 7fc644fd0640 1 -- 172.21.15.170:0/1081646111 wait complete. 2024-05-08T04:13:19.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.005+0000 7fc644fd0640 1 Processor -- start 2024-05-08T04:13:19.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.005+0000 7fc644fd0640 1 -- start start 2024-05-08T04:13:19.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.005+0000 7fc644fd0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc640070b70 0x7fc6400b1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.005+0000 7fc644fd0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc640143260 0x7fc6400b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.005+0000 7fc644fd0640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc640147520 0x7fc6400b4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.009 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.005+0000 7fc644fd0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc640143eb0 con 0x7fc640070b70 2024-05-08T04:13:19.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.005+0000 7fc644fd0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc640143d30 con 0x7fc640143260 2024-05-08T04:13:19.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.005+0000 7fc644fd0640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fc640144030 con 0x7fc640147520 2024-05-08T04:13:19.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.005+0000 7fc63e59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc640070b70 0x7fc6400b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.005+0000 7fc63ed9d640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc640147520 0x7fc6400b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.005+0000 7fc63e59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc640070b70 0x7fc6400b1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49486/0 (socket says 172.21.15.170:49486) 2024-05-08T04:13:19.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.005+0000 7fc63e59c640 1 -- 172.21.15.170:0/469188926 learned_addr learned my addr 172.21.15.170:0/469188926 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:13:19.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.005+0000 7fc637fff640 1 --2- 172.21.15.170:0/469188926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc640143260 0x7fc6400b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.006+0000 7fc63ed9d640 1 -- 172.21.15.170:0/469188926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc640143260 msgr2=0x7fc6400b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:19.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.006+0000 7fc63ed9d640 1 --2- 172.21.15.170:0/469188926 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc640143260 0x7fc6400b1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.006+0000 7fc63ed9d640 1 -- 172.21.15.170:0/469188926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc640070b70 msgr2=0x7fc6400b1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:19.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.006+0000 7fc63ed9d640 1 --2- 172.21.15.170:0/469188926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc640070b70 0x7fc6400b1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.006+0000 7fc63ed9d640 1 -- 172.21.15.170:0/469188926 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6400b5f50 con 0x7fc640147520 2024-05-08T04:13:19.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.006+0000 7fc63e59c640 1 --2- 172.21.15.170:0/469188926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc640070b70 0x7fc6400b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:13:19.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.006+0000 7fc63ed9d640 1 --2- 172.21.15.170:0/469188926 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc640147520 0x7fc6400b4250 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc638032000 tx=0x7fc638004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:19.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.006+0000 7fc636ffd640 1 -- 172.21.15.170:0/469188926 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc638043020 con 0x7fc640147520 2024-05-08T04:13:19.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.006+0000 7fc636ffd640 1 -- 172.21.15.170:0/469188926 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc63803e450 con 0x7fc640147520 2024-05-08T04:13:19.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.006+0000 7fc644fd0640 1 -- 172.21.15.170:0/469188926 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6400b5a00 con 0x7fc640147520 2024-05-08T04:13:19.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.006+0000 7fc636ffd640 1 -- 172.21.15.170:0/469188926 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc638009510 con 0x7fc640147520 2024-05-08T04:13:19.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.006+0000 7fc644fd0640 1 -- 172.21.15.170:0/469188926 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc64013ce90 con 0x7fc640147520 2024-05-08T04:13:19.011 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.007+0000 7fc644fd0640 1 -- 172.21.15.170:0/469188926 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc6400b2b50 con 0x7fc640147520 2024-05-08T04:13:19.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.008+0000 7fc636ffd640 1 -- 172.21.15.170:0/469188926 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7fc638046020 con 0x7fc640147520 2024-05-08T04:13:19.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.008+0000 7fc636ffd640 1 --2- 172.21.15.170:0/469188926 >> v2:172.21.15.187:6816/101007897 conn(0x7fc620076290 0x7fc620078750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.008+0000 7fc636ffd640 1 -- 172.21.15.170:0/469188926 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 4525+0+0 (secure 0 0 0) 0x7fc6380b9640 con 0x7fc640147520 2024-05-08T04:13:19.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.008+0000 7fc63e59c640 1 --2- 172.21.15.170:0/469188926 >> v2:172.21.15.187:6816/101007897 conn(0x7fc620076290 0x7fc620078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.008+0000 7fc636ffd640 1 --2- 172.21.15.170:0/469188926 >> v2:172.21.15.187:6808/1425849822 conn(0x7fc62007cff0 0x7fc62007f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.008+0000 7fc636ffd640 1 -- 172.21.15.170:0/469188926 --> v2:172.21.15.187:6808/1425849822 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc63803be50 con 0x7fc62007cff0 2024-05-08T04:13:19.012 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.008+0000 7fc636ffd640 1 -- 172.21.15.170:0/469188926 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc6380c0050 con 0x7fc640147520 2024-05-08T04:13:19.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.008+0000 7fc637fff640 1 --2- 172.21.15.170:0/469188926 >> v2:172.21.15.187:6808/1425849822 conn(0x7fc62007cff0 0x7fc62007f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.009+0000 7fc63e59c640 1 --2- 172.21.15.170:0/469188926 >> v2:172.21.15.187:6816/101007897 conn(0x7fc620076290 0x7fc620078750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc630009ed0 tx=0x7fc630009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:19.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.009+0000 7fc637fff640 1 --2- 172.21.15.170:0/469188926 >> v2:172.21.15.187:6808/1425849822 conn(0x7fc62007cff0 0x7fc62007f410 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:19.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.009+0000 7fc636ffd640 1 -- 172.21.15.170:0/469188926 <== osd.4 v2:172.21.15.187:6808/1425849822 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc63803be50 con 0x7fc62007cff0 2024-05-08T04:13:19.024 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.023+0000 7fc644fd0640 1 -- 172.21.15.170:0/469188926 --> v2:172.21.15.187:6808/1425849822 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc6400b2d90 con 0x7fc62007cff0 2024-05-08T04:13:19.025 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.023+0000 7fc636ffd640 1 -- 172.21.15.170:0/469188926 <== osd.4 v2:172.21.15.187:6808/1425849822 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc6400b2d90 con 0x7fc62007cff0 2024-05-08T04:13:19.025 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-05-08T04:13:19.025 INFO:teuthology.orchestra.run.smithi170.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T04:13:19.025 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-05-08T04:13:19.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.023+0000 7fc644fd0640 1 -- 172.21.15.170:0/469188926 >> v2:172.21.15.187:6808/1425849822 conn(0x7fc62007cff0 msgr2=0x7fc62007f410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:19.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.023+0000 7fc644fd0640 1 --2- 172.21.15.170:0/469188926 >> v2:172.21.15.187:6808/1425849822 conn(0x7fc62007cff0 0x7fc62007f410 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.023+0000 7fc644fd0640 1 -- 172.21.15.170:0/469188926 >> v2:172.21.15.187:6816/101007897 conn(0x7fc620076290 msgr2=0x7fc620078750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:19.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.023+0000 7fc644fd0640 1 --2- 172.21.15.170:0/469188926 >> v2:172.21.15.187:6816/101007897 conn(0x7fc620076290 0x7fc620078750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc630009ed0 tx=0x7fc630009190 comp rx=0 tx=0).stop 2024-05-08T04:13:19.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.023+0000 7fc644fd0640 1 -- 172.21.15.170:0/469188926 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc640147520 msgr2=0x7fc6400b4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:19.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.023+0000 7fc644fd0640 1 --2- 172.21.15.170:0/469188926 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc640147520 0x7fc6400b4250 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc638032000 tx=0x7fc638004a40 comp rx=0 tx=0).stop 2024-05-08T04:13:19.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.023+0000 7fc63ed9d640 1 -- 172.21.15.170:0/469188926 reap_dead start 2024-05-08T04:13:19.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.023+0000 7fc644fd0640 1 -- 172.21.15.170:0/469188926 shutdown_connections 2024-05-08T04:13:19.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.023+0000 7fc644fd0640 1 -- 172.21.15.170:0/469188926 >> 172.21.15.170:0/469188926 conn(0x7fc6400b64d0 msgr2=0x7fc6400b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:19.026 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.024+0000 7fc644fd0640 1 -- 172.21.15.170:0/469188926 shutdown_connections 2024-05-08T04:13:19.027 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.024+0000 7fc644fd0640 1 -- 172.21.15.170:0/469188926 wait complete. 2024-05-08T04:13:19.037 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-08T04:13:19.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.140+0000 7fcfc0e8e640 1 Processor -- start 2024-05-08T04:13:19.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.140+0000 7fcfc0e8e640 1 -- start start 2024-05-08T04:13:19.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.141+0000 7fcfc0e8e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fcfbc0b6e50 0x7fcfbc143ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.141+0000 7fcfc0e8e640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcfbc072bc0 0x7fcfbc147ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.141+0000 7fcfc0e8e640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfbc0b7230 con 0x7fcfbc148020 2024-05-08T04:13:19.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.141+0000 7fcfc0e8e640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7fcfbc074950 con 0x7fcfbc072bc0 2024-05-08T04:13:19.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.141+0000 7fcfc0e8e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fcfbc1475c0 con 0x7fcfbc0b6e50 2024-05-08T04:13:19.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.141+0000 7fcfb9d9b640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcfbc072bc0 0x7fcfbc147ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.141+0000 7fcfb9d9b640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcfbc072bc0 0x7fcfbc147ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49488/0 (socket says 172.21.15.170:49488) 2024-05-08T04:13:19.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.141+0000 7fcfbad9d640 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fcfbc148020 0x7fcfbc143320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43770/0 (socket says 172.21.15.170:43770) 2024-05-08T04:13:19.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.141+0000 7fcfb9d9b640 1 -- 172.21.15.170:0/3808978244 learned_addr learned my addr 172.21.15.170:0/3808978244 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:13:19.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.141+0000 7fcfba59c640 1 --2- 172.21.15.170:0/3808978244 >> v2:172.21.15.187:3300/0 conn(0x7fcfbc0b6e50 0x7fcfbc143ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.141+0000 7fcfb9d9b640 1 -- 172.21.15.170:0/3808978244 >> v2:172.21.15.187:3300/0 conn(0x7fcfbc0b6e50 msgr2=0x7fcfbc143ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:13:19.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.141+0000 7fcfb9d9b640 1 --2- 172.21.15.170:0/3808978244 >> v2:172.21.15.187:3300/0 conn(0x7fcfbc0b6e50 0x7fcfbc143ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.141+0000 7fcfb9d9b640 1 -- 172.21.15.170:0/3808978244 >> v1:172.21.15.170:6789/0 conn(0x7fcfbc148020 legacy=0x7fcfbc143320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:19.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.141+0000 7fcfb9d9b640 1 -- 172.21.15.170:0/3808978244 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfbc073000 con 0x7fcfbc072bc0 2024-05-08T04:13:19.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.141+0000 7fcfb959a640 1 -- 172.21.15.170:0/3808978244 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1504930294 0 0) 0x7fcfbc0b7230 con 0x7fcfbc148020 2024-05-08T04:13:19.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.142+0000 7fcfb9d9b640 1 --2- 172.21.15.170:0/3808978244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfbc072bc0 0x7fcfbc147ae0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc0ba870 tx=0x7fcfa402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72383745c88a89a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:19.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.142+0000 7fcfb959a640 1 -- 172.21.15.170:0/3808978244 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4034070 con 0x7fcfbc072bc0 2024-05-08T04:13:19.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.142+0000 7fcfb959a640 1 -- 172.21.15.170:0/3808978244 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfa4030860 con 0x7fcfbc072bc0 2024-05-08T04:13:19.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.142+0000 7fcfb959a640 1 -- 172.21.15.170:0/3808978244 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfa4030c30 con 0x7fcfbc072bc0 2024-05-08T04:13:19.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.143+0000 7fcfc0e8e640 1 -- 172.21.15.170:0/3808978244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfbc072bc0 msgr2=0x7fcfbc147ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:19.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.143+0000 7fcfc0e8e640 1 --2- 172.21.15.170:0/3808978244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfbc072bc0 0x7fcfbc147ae0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc0ba870 tx=0x7fcfa402b9a0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.143+0000 7fcfc0e8e640 1 -- 172.21.15.170:0/3808978244 shutdown_connections 2024-05-08T04:13:19.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.143+0000 7fcfc0e8e640 1 --2- 172.21.15.170:0/3808978244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfbc072bc0 0x7fcfbc147ae0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.143+0000 7fcfc0e8e640 1 --2- 172.21.15.170:0/3808978244 >> v2:172.21.15.187:3300/0 conn(0x7fcfbc0b6e50 0x7fcfbc143ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.143+0000 7fcfc0e8e640 1 -- 172.21.15.170:0/3808978244 >> 172.21.15.170:0/3808978244 conn(0x7fcfbc0ba000 msgr2=0x7fcfbc0b64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:19.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.143+0000 7fcfc0e8e640 1 -- 172.21.15.170:0/3808978244 shutdown_connections 2024-05-08T04:13:19.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.143+0000 7fcfc0e8e640 1 -- 172.21.15.170:0/3808978244 wait complete. 2024-05-08T04:13:19.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.144+0000 7fcfc0e8e640 1 Processor -- start 2024-05-08T04:13:19.146 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.144+0000 7fcfc0e8e640 1 -- start start 2024-05-08T04:13:19.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.145+0000 7fcfc0e8e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcfbc072bc0 0x7fcfbc13d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.145+0000 7fcfc0e8e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfbc0b6e50 0x7fcfbc13cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.145+0000 7fcfc0e8e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfbc148020 0x7fcfbc138e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.145+0000 7fcfc0e8e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc143bb0 con 0x7fcfbc148020 2024-05-08T04:13:19.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.145+0000 7fcfc0e8e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc143a30 con 0x7fcfbc0b6e50 2024-05-08T04:13:19.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.145+0000 7fcfc0e8e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fcfbc14e7b0 con 0x7fcfbc072bc0 2024-05-08T04:13:19.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.145+0000 7fcfba59c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcfbc072bc0 0x7fcfbc13d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.145+0000 7fcfbad9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfbc148020 0x7fcfbc138e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.145+0000 7fcfba59c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcfbc072bc0 0x7fcfbc13d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:48126/0 (socket says 172.21.15.170:48126) 2024-05-08T04:13:19.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.145+0000 7fcfba59c640 1 -- 172.21.15.170:0/577824721 learned_addr learned my addr 172.21.15.170:0/577824721 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:13:19.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.145+0000 7fcfb9d9b640 1 --2- 172.21.15.170:0/577824721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfbc0b6e50 0x7fcfbc13cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.145+0000 7fcfbad9d640 1 -- 172.21.15.170:0/577824721 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcfbc072bc0 msgr2=0x7fcfbc13d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:19.149 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.145+0000 7fcfbad9d640 1 --2- 172.21.15.170:0/577824721 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcfbc072bc0 0x7fcfbc13d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.145+0000 7fcfbad9d640 1 -- 172.21.15.170:0/577824721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfbc0b6e50 msgr2=0x7fcfbc13cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:19.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.145+0000 7fcfbad9d640 1 --2- 172.21.15.170:0/577824721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfbc0b6e50 0x7fcfbc13cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.145+0000 7fcfbad9d640 1 -- 172.21.15.170:0/577824721 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfbc13ef00 con 0x7fcfbc148020 2024-05-08T04:13:19.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.145+0000 7fcfb9d9b640 1 --2- 172.21.15.170:0/577824721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfbc0b6e50 0x7fcfbc13cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:13:19.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.145+0000 7fcfba59c640 1 --2- 172.21.15.170:0/577824721 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcfbc072bc0 0x7fcfbc13d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:13:19.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.145+0000 7fcfbad9d640 1 --2- 172.21.15.170:0/577824721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfbc148020 0x7fcfbc138e60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400c700 tx=0x7fcfb400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:19.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.145+0000 7fcfb2ffd640 1 -- 172.21.15.170:0/577824721 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4018070 con 0x7fcfbc148020 2024-05-08T04:13:19.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.145+0000 7fcfb2ffd640 1 -- 172.21.15.170:0/577824721 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfb4004d60 con 0x7fcfbc148020 2024-05-08T04:13:19.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.145+0000 7fcfb2ffd640 1 -- 172.21.15.170:0/577824721 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb400a820 con 0x7fcfbc148020 2024-05-08T04:13:19.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.146+0000 7fcfc0e8e640 1 -- 172.21.15.170:0/577824721 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfbc13f1e0 con 0x7fcfbc148020 2024-05-08T04:13:19.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.146+0000 7fcfc0e8e640 1 -- 172.21.15.170:0/577824721 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfbc13d1c0 con 0x7fcfbc148020 2024-05-08T04:13:19.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.146+0000 7fcfc0e8e640 1 -- 172.21.15.170:0/577824721 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfbc06e9e0 con 0x7fcfbc148020 2024-05-08T04:13:19.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.147+0000 7fcfb2ffd640 1 -- 172.21.15.170:0/577824721 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7fcfb401b070 con 0x7fcfbc148020 2024-05-08T04:13:19.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.148+0000 7fcfb2ffd640 1 --2- 172.21.15.170:0/577824721 >> v2:172.21.15.187:6816/101007897 conn(0x7fcf9c0762c0 0x7fcf9c078780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.148+0000 7fcfba59c640 1 --2- 172.21.15.170:0/577824721 >> v2:172.21.15.187:6816/101007897 conn(0x7fcf9c0762c0 0x7fcf9c078780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.148+0000 7fcfb2ffd640 1 -- 172.21.15.170:0/577824721 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 4525+0+0 (secure 0 0 0) 0x7fcfb4096410 con 0x7fcfbc148020 2024-05-08T04:13:19.151 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.148+0000 7fcfba59c640 1 --2- 172.21.15.170:0/577824721 >> v2:172.21.15.187:6816/101007897 conn(0x7fcf9c0762c0 0x7fcf9c078780 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcfac006fd0 tx=0x7fcfac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:19.154 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.152+0000 7fcfb2ffd640 1 -- 172.21.15.170:0/577824721 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fcfb405fc50 con 0x7fcfbc148020 2024-05-08T04:13:19.356 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.355+0000 7fcfc0e8e640 1 -- 172.21.15.170:0/577824721 --> v2:172.21.15.187:6816/101007897 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fcfbc071570 con 0x7fcf9c0762c0 2024-05-08T04:13:19.357 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.355+0000 7fcfb2ffd640 1 -- 172.21.15.170:0/577824721 <== mgr.4107 v2:172.21.15.187:6816/101007897 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fcfbc071570 con 0x7fcf9c0762c0 2024-05-08T04:13:19.357 INFO:teuthology.orchestra.run.smithi170.stderr:instructed osd(s) 4 to deep-scrub 2024-05-08T04:13:19.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.357+0000 7fcfc0e8e640 1 -- 172.21.15.170:0/577824721 >> v2:172.21.15.187:6816/101007897 conn(0x7fcf9c0762c0 msgr2=0x7fcf9c078780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:19.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.357+0000 7fcfc0e8e640 1 --2- 172.21.15.170:0/577824721 >> v2:172.21.15.187:6816/101007897 conn(0x7fcf9c0762c0 0x7fcf9c078780 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcfac006fd0 tx=0x7fcfac008040 comp rx=0 tx=0).stop 2024-05-08T04:13:19.359 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.357+0000 7fcfc0e8e640 1 -- 172.21.15.170:0/577824721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfbc148020 msgr2=0x7fcfbc138e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:19.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.357+0000 7fcfc0e8e640 1 --2- 172.21.15.170:0/577824721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfbc148020 0x7fcfbc138e60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400c700 tx=0x7fcfb400cbc0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.358+0000 7fcfc0e8e640 1 -- 172.21.15.170:0/577824721 shutdown_connections 2024-05-08T04:13:19.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.358+0000 7fcfc0e8e640 1 --2- 172.21.15.170:0/577824721 >> v2:172.21.15.187:6816/101007897 conn(0x7fcf9c0762c0 0x7fcf9c078780 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.358+0000 7fcfc0e8e640 1 --2- 172.21.15.170:0/577824721 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcfbc0b6e50 0x7fcfbc13cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.358+0000 7fcfc0e8e640 1 --2- 172.21.15.170:0/577824721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcfbc148020 0x7fcfbc138e60 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.358+0000 7fcfc0e8e640 1 --2- 172.21.15.170:0/577824721 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcfbc072bc0 0x7fcfbc13d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.358+0000 7fcfc0e8e640 1 -- 172.21.15.170:0/577824721 >> 172.21.15.170:0/577824721 conn(0x7fcfbc0ba000 msgr2=0x7fcfbc13b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:19.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.358+0000 7fcfc0e8e640 1 -- 172.21.15.170:0/577824721 shutdown_connections 2024-05-08T04:13:19.360 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.358+0000 7fcfc0e8e640 1 -- 172.21.15.170:0/577824721 wait complete. 2024-05-08T04:13:19.373 INFO:tasks.ceph:Scrubbing osd.5 2024-05-08T04:13:19.374 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-08T04:13:19.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.487+0000 7f7ac6902640 1 Processor -- start 2024-05-08T04:13:19.489 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.487+0000 7f7ac6902640 1 -- start start 2024-05-08T04:13:19.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.487+0000 7f7ac6902640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ac00aafc0 0x7f7ac0071c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.487+0000 7f7ac6902640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7ac01425f0 0x7f7ac013b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.487+0000 7f7ac6902640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7ac013bbe0 0x7f7ac0145270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.487+0000 7f7ac6902640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7f7ac00ab3a0 con 0x7f7ac013bbe0 2024-05-08T04:13:19.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.487+0000 7f7ac6902640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f7ac00ac780 con 0x7f7ac01425f0 2024-05-08T04:13:19.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.487+0000 7f7ac6902640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f7ac00aca60 con 0x7f7ac00aafc0 2024-05-08T04:13:19.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.488+0000 7f7ac4e9f640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7ac013bbe0 0x7f7ac0145270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.492 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.488+0000 7f7abffff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ac00aafc0 0x7f7ac0071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.488+0000 7f7ac4e9f640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f7ac013bbe0 0x7f7ac0145270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49512/0 (socket says 172.21.15.170:49512) 2024-05-08T04:13:19.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.488+0000 7f7abffff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ac00aafc0 0x7f7ac0071c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:48128/0 (socket says 172.21.15.170:48128) 2024-05-08T04:13:19.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.488+0000 7f7ac4e9f640 1 -- 172.21.15.170:0/2957554951 learned_addr learned my addr 172.21.15.170:0/2957554951 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:13:19.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.488+0000 7f7ab7fff640 1 --2- 172.21.15.170:0/2957554951 >> v2:172.21.15.187:3300/0 conn(0x7f7ac01425f0 0x7f7ac013b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.488+0000 7f7ac4e9f640 1 -- 172.21.15.170:0/2957554951 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ac00aafc0 msgr2=0x7f7ac0071c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:13:19.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.488+0000 7f7ac4e9f640 1 --2- 172.21.15.170:0/2957554951 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ac00aafc0 0x7f7ac0071c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.488+0000 7f7ac4e9f640 1 -- 172.21.15.170:0/2957554951 >> v2:172.21.15.187:3300/0 conn(0x7f7ac01425f0 msgr2=0x7f7ac013b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:13:19.493 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.488+0000 7f7ac4e9f640 1 --2- 172.21.15.170:0/2957554951 >> v2:172.21.15.187:3300/0 conn(0x7f7ac01425f0 0x7f7ac013b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.488+0000 7f7ac4e9f640 1 -- 172.21.15.170:0/2957554951 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac01420d0 con 0x7f7ac013bbe0 2024-05-08T04:13:19.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.488+0000 7f7abffff640 1 --2- 172.21.15.170:0/2957554951 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ac00aafc0 0x7f7ac0071c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:13:19.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.489+0000 7f7ac4e9f640 1 --2- 172.21.15.170:0/2957554951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ac013bbe0 0x7f7ac0145270 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7ac00711c0 tx=0x7f7ab802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7108937d0ff55552 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:19.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.489+0000 7f7abf7fe640 1 -- 172.21.15.170:0/2957554951 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab802c9e0 con 0x7f7ac013bbe0 2024-05-08T04:13:19.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.489+0000 7f7abf7fe640 1 -- 172.21.15.170:0/2957554951 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ab802cb80 con 0x7f7ac013bbe0 2024-05-08T04:13:19.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.489+0000 7f7abf7fe640 1 -- 172.21.15.170:0/2957554951 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab80343f0 con 0x7f7ac013bbe0 2024-05-08T04:13:19.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.489+0000 7f7ac6902640 1 -- 172.21.15.170:0/2957554951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ac013bbe0 msgr2=0x7f7ac0145270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:19.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.489+0000 7f7ac6902640 1 --2- 172.21.15.170:0/2957554951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ac013bbe0 0x7f7ac0145270 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7ac00711c0 tx=0x7f7ab802be40 comp rx=0 tx=0).stop 2024-05-08T04:13:19.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.489+0000 7f7ac6902640 1 -- 172.21.15.170:0/2957554951 shutdown_connections 2024-05-08T04:13:19.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.489+0000 7f7ac6902640 1 --2- 172.21.15.170:0/2957554951 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ac013bbe0 0x7f7ac0145270 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.489+0000 7f7ac6902640 1 --2- 172.21.15.170:0/2957554951 >> v2:172.21.15.187:3300/0 conn(0x7f7ac01425f0 0x7f7ac013b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.489+0000 7f7ac6902640 1 --2- 172.21.15.170:0/2957554951 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ac00aafc0 0x7f7ac0071c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.489+0000 7f7ac6902640 1 -- 172.21.15.170:0/2957554951 >> 172.21.15.170:0/2957554951 conn(0x7f7ac00b7990 msgr2=0x7f7ac00b7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:19.494 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.489+0000 7f7ac6902640 1 -- 172.21.15.170:0/2957554951 shutdown_connections 2024-05-08T04:13:19.495 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.489+0000 7f7ac6902640 1 -- 172.21.15.170:0/2957554951 wait complete. 2024-05-08T04:13:19.497 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.490+0000 7f7ac6902640 1 Processor -- start 2024-05-08T04:13:19.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.490+0000 7f7ac6902640 1 -- start start 2024-05-08T04:13:19.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.491+0000 7f7ac6902640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ac00aafc0 0x7f7ac00b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.491+0000 7f7ac6902640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ac013bbe0 0x7f7ac0134b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.491+0000 7f7ac6902640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ac01425f0 0x7f7ac0135050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.491+0000 7f7ac6902640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac013c030 con 0x7f7ac01425f0 2024-05-08T04:13:19.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.491+0000 7f7ac6902640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac00aa980 con 0x7f7ac013bbe0 2024-05-08T04:13:19.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.491+0000 7f7ac6902640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f7ac0140630 con 0x7f7ac00aafc0 2024-05-08T04:13:19.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.491+0000 7f7abffff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ac00aafc0 0x7f7ac00b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.491+0000 7f7abffff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ac00aafc0 0x7f7ac00b17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:48142/0 (socket says 172.21.15.170:48142) 2024-05-08T04:13:19.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.491+0000 7f7ac4e9f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ac01425f0 0x7f7ac0135050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.491+0000 7f7abffff640 1 -- 172.21.15.170:0/2322280657 learned_addr learned my addr 172.21.15.170:0/2322280657 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:13:19.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.491+0000 7f7ac4e9f640 1 -- 172.21.15.170:0/2322280657 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ac00aafc0 msgr2=0x7f7ac00b17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:19.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.491+0000 7f7ac4e9f640 1 --2- 172.21.15.170:0/2322280657 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7ac00aafc0 0x7f7ac00b17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.498 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.491+0000 7f7ac4e9f640 1 -- 172.21.15.170:0/2322280657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ac013bbe0 msgr2=0x7f7ac0134b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:19.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.491+0000 7f7ac4e9f640 1 --2- 172.21.15.170:0/2322280657 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ac013bbe0 0x7f7ac0134b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.491+0000 7f7ac4e9f640 1 -- 172.21.15.170:0/2322280657 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac0135590 con 0x7f7ac01425f0 2024-05-08T04:13:19.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.491+0000 7f7ac4e9f640 1 --2- 172.21.15.170:0/2322280657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ac01425f0 0x7f7ac0135050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8032000 tx=0x7f7ab8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:19.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.492+0000 7f7abd7fa640 1 -- 172.21.15.170:0/2322280657 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8043020 con 0x7f7ac01425f0 2024-05-08T04:13:19.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.492+0000 7f7abd7fa640 1 -- 172.21.15.170:0/2322280657 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ab803e450 con 0x7f7ac01425f0 2024-05-08T04:13:19.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.492+0000 7f7abd7fa640 1 -- 172.21.15.170:0/2322280657 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab8009510 con 0x7f7ac01425f0 2024-05-08T04:13:19.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.492+0000 7f7ac6902640 1 -- 172.21.15.170:0/2322280657 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ac00b1d20 con 0x7f7ac01425f0 2024-05-08T04:13:19.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.492+0000 7f7ac6902640 1 -- 172.21.15.170:0/2322280657 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ac01412f0 con 0x7f7ac01425f0 2024-05-08T04:13:19.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.492+0000 7f7ac6902640 1 -- 172.21.15.170:0/2322280657 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7ac006d400 con 0x7f7ac01425f0 2024-05-08T04:13:19.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.493+0000 7f7abd7fa640 1 -- 172.21.15.170:0/2322280657 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f7ab8005d10 con 0x7f7ac01425f0 2024-05-08T04:13:19.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.494+0000 7f7abd7fa640 1 --2- 172.21.15.170:0/2322280657 >> v2:172.21.15.187:6816/101007897 conn(0x7f7aa0076290 0x7f7aa0078750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.494+0000 7f7abd7fa640 1 -- 172.21.15.170:0/2322280657 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 4525+0+0 (secure 0 0 0) 0x7f7ab80b9980 con 0x7f7ac01425f0 2024-05-08T04:13:19.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.494+0000 7f7abffff640 1 --2- 172.21.15.170:0/2322280657 >> v2:172.21.15.187:6816/101007897 conn(0x7f7aa0076290 0x7f7aa0078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.494+0000 7f7abd7fa640 1 --2- 172.21.15.170:0/2322280657 >> v2:172.21.15.187:6800/3273856499 conn(0x7f7aa007cff0 0x7f7aa007f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.499 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.494+0000 7f7abd7fa640 1 -- 172.21.15.170:0/2322280657 --> v2:172.21.15.187:6800/3273856499 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7ab8041e50 con 0x7f7aa007cff0 2024-05-08T04:13:19.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.494+0000 7f7abd7fa640 1 -- 172.21.15.170:0/2322280657 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f7ab80be2c0 con 0x7f7ac01425f0 2024-05-08T04:13:19.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.494+0000 7f7ab7fff640 1 --2- 172.21.15.170:0/2322280657 >> v2:172.21.15.187:6800/3273856499 conn(0x7f7aa007cff0 0x7f7aa007f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.494+0000 7f7abffff640 1 --2- 172.21.15.170:0/2322280657 >> v2:172.21.15.187:6816/101007897 conn(0x7f7aa0076290 0x7f7aa0078750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0009560 tx=0x7f7ab0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:19.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.494+0000 7f7ab7fff640 1 --2- 172.21.15.170:0/2322280657 >> v2:172.21.15.187:6800/3273856499 conn(0x7f7aa007cff0 0x7f7aa007f410 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:19.500 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.495+0000 7f7abd7fa640 1 -- 172.21.15.170:0/2322280657 <== osd.5 v2:172.21.15.187:6800/3273856499 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7ab8041e50 con 0x7f7aa007cff0 2024-05-08T04:13:19.509 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.508+0000 7f7ac6902640 1 -- 172.21.15.170:0/2322280657 --> v2:172.21.15.187:6800/3273856499 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7ac006e9e0 con 0x7f7aa007cff0 2024-05-08T04:13:19.510 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.508+0000 7f7abd7fa640 1 -- 172.21.15.170:0/2322280657 <== osd.5 v2:172.21.15.187:6800/3273856499 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7ac006e9e0 con 0x7f7aa007cff0 2024-05-08T04:13:19.510 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-05-08T04:13:19.510 INFO:teuthology.orchestra.run.smithi170.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T04:13:19.510 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-05-08T04:13:19.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.509+0000 7f7ac6902640 1 -- 172.21.15.170:0/2322280657 >> v2:172.21.15.187:6800/3273856499 conn(0x7f7aa007cff0 msgr2=0x7f7aa007f410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:19.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.509+0000 7f7ac6902640 1 --2- 172.21.15.170:0/2322280657 >> v2:172.21.15.187:6800/3273856499 conn(0x7f7aa007cff0 0x7f7aa007f410 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.509+0000 7f7ac6902640 1 -- 172.21.15.170:0/2322280657 >> v2:172.21.15.187:6816/101007897 conn(0x7f7aa0076290 msgr2=0x7f7aa0078750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:19.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.509+0000 7f7ac6902640 1 --2- 172.21.15.170:0/2322280657 >> v2:172.21.15.187:6816/101007897 conn(0x7f7aa0076290 0x7f7aa0078750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0009560 tx=0x7f7ab0009190 comp rx=0 tx=0).stop 2024-05-08T04:13:19.511 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.509+0000 7f7ac6902640 1 -- 172.21.15.170:0/2322280657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ac01425f0 msgr2=0x7f7ac0135050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:19.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.509+0000 7f7ac6902640 1 --2- 172.21.15.170:0/2322280657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ac01425f0 0x7f7ac0135050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8032000 tx=0x7f7ab8004a40 comp rx=0 tx=0).stop 2024-05-08T04:13:19.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.509+0000 7f7ac4e9f640 1 -- 172.21.15.170:0/2322280657 reap_dead start 2024-05-08T04:13:19.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.509+0000 7f7ac6902640 1 -- 172.21.15.170:0/2322280657 shutdown_connections 2024-05-08T04:13:19.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.509+0000 7f7ac6902640 1 -- 172.21.15.170:0/2322280657 >> 172.21.15.170:0/2322280657 conn(0x7f7ac00b7990 msgr2=0x7f7ac0072420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:19.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.509+0000 7f7ac6902640 1 -- 172.21.15.170:0/2322280657 shutdown_connections 2024-05-08T04:13:19.512 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.509+0000 7f7ac6902640 1 -- 172.21.15.170:0/2322280657 wait complete. 2024-05-08T04:13:19.521 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-08T04:13:19.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.618+0000 7ff8f81dd640 1 Processor -- start 2024-05-08T04:13:19.620 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.619+0000 7ff8f81dd640 1 -- start start 2024-05-08T04:13:19.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.619+0000 7ff8f81dd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff8f01480a0 0x7ff8f006fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.619+0000 7ff8f81dd640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff8f01479b0 0x7ff8f00701e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.619+0000 7ff8f81dd640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8f0070720 con 0x7ff8f01475d0 2024-05-08T04:13:19.622 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.619+0000 7ff8f81dd640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7ff8f0074350 con 0x7ff8f01479b0 2024-05-08T04:13:19.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.619+0000 7ff8f81dd640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff8f0074610 con 0x7ff8f01480a0 2024-05-08T04:13:19.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.619+0000 7ff8f677a640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff8f01479b0 0x7ff8f00701e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.619+0000 7ff8f677a640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff8f01479b0 0x7ff8f00701e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49536/0 (socket says 172.21.15.170:49536) 2024-05-08T04:13:19.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.619+0000 7ff8f5f79640 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7ff8f01475d0 0x7ff8f0143190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43776/0 (socket says 172.21.15.170:43776) 2024-05-08T04:13:19.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.619+0000 7ff8f677a640 1 -- 172.21.15.170:0/2920110398 learned_addr learned my addr 172.21.15.170:0/2920110398 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:13:19.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.620+0000 7ff8f5778640 1 --2- 172.21.15.170:0/2920110398 >> v2:172.21.15.187:3300/0 conn(0x7ff8f01480a0 0x7ff8f006fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.620+0000 7ff8f677a640 1 -- 172.21.15.170:0/2920110398 >> v2:172.21.15.187:3300/0 conn(0x7ff8f01480a0 msgr2=0x7ff8f006fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:13:19.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.620+0000 7ff8f677a640 1 --2- 172.21.15.170:0/2920110398 >> v2:172.21.15.187:3300/0 conn(0x7ff8f01480a0 0x7ff8f006fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.620+0000 7ff8f677a640 1 -- 172.21.15.170:0/2920110398 >> v1:172.21.15.170:6789/0 conn(0x7ff8f01475d0 legacy=0x7ff8f0143190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:19.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.620+0000 7ff8f677a640 1 -- 172.21.15.170:0/2920110398 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f00748f0 con 0x7ff8f01479b0 2024-05-08T04:13:19.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.620+0000 7ff8f4f77640 1 -- 172.21.15.170:0/2920110398 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 475231940 0 0) 0x7ff8f0070720 con 0x7ff8f01475d0 2024-05-08T04:13:19.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.620+0000 7ff8f677a640 1 --2- 172.21.15.170:0/2920110398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8f01479b0 0x7ff8f00701e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0071650 tx=0x7ff8e002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77bc70e68a405931 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:19.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.621+0000 7ff8f4f77640 1 -- 172.21.15.170:0/2920110398 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0035020 con 0x7ff8f01479b0 2024-05-08T04:13:19.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.621+0000 7ff8f4f77640 1 -- 172.21.15.170:0/2920110398 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8e0031870 con 0x7ff8f01479b0 2024-05-08T04:13:19.623 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.621+0000 7ff8f4f77640 1 -- 172.21.15.170:0/2920110398 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0031c40 con 0x7ff8f01479b0 2024-05-08T04:13:19.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.621+0000 7ff8f81dd640 1 -- 172.21.15.170:0/2920110398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8f01479b0 msgr2=0x7ff8f00701e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:19.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.621+0000 7ff8f81dd640 1 --2- 172.21.15.170:0/2920110398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8f01479b0 0x7ff8f00701e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0071650 tx=0x7ff8e002b910 comp rx=0 tx=0).stop 2024-05-08T04:13:19.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.621+0000 7ff8f81dd640 1 -- 172.21.15.170:0/2920110398 shutdown_connections 2024-05-08T04:13:19.626 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.621+0000 7ff8f81dd640 1 --2- 172.21.15.170:0/2920110398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8f01479b0 0x7ff8f00701e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.621+0000 7ff8f81dd640 1 --2- 172.21.15.170:0/2920110398 >> v2:172.21.15.187:3300/0 conn(0x7ff8f01480a0 0x7ff8f006fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.621+0000 7ff8f81dd640 1 -- 172.21.15.170:0/2920110398 >> 172.21.15.170:0/2920110398 conn(0x7ff8f00b9e60 msgr2=0x7ff8f00ba270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:19.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.621+0000 7ff8f81dd640 1 -- 172.21.15.170:0/2920110398 shutdown_connections 2024-05-08T04:13:19.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.621+0000 7ff8f81dd640 1 -- 172.21.15.170:0/2920110398 wait complete. 2024-05-08T04:13:19.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.622+0000 7ff8f81dd640 1 Processor -- start 2024-05-08T04:13:19.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.622+0000 7ff8f81dd640 1 -- start start 2024-05-08T04:13:19.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.623+0000 7ff8f81dd640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff8f01475d0 0x7ff8f013cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.623+0000 7ff8f81dd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8f01479b0 0x7ff8f00b1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.623+0000 7ff8f81dd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8f00b02d0 0x7ff8f013af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.623+0000 7ff8f81dd640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f01438a0 con 0x7ff8f00b02d0 2024-05-08T04:13:19.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.623+0000 7ff8f81dd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f00ae980 con 0x7ff8f01479b0 2024-05-08T04:13:19.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.623+0000 7ff8f81dd640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7ff8f0143a20 con 0x7ff8f01475d0 2024-05-08T04:13:19.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.623+0000 7ff8f677a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8f00b02d0 0x7ff8f013af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.623+0000 7ff8f5f79640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff8f01475d0 0x7ff8f013cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.627 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.623+0000 7ff8f677a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8f00b02d0 0x7ff8f013af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49546/0 (socket says 172.21.15.170:49546) 2024-05-08T04:13:19.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.623+0000 7ff8f677a640 1 -- 172.21.15.170:0/1223643684 learned_addr learned my addr 172.21.15.170:0/1223643684 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:13:19.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.623+0000 7ff8f5778640 1 --2- 172.21.15.170:0/1223643684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8f01479b0 0x7ff8f00b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.623+0000 7ff8f677a640 1 -- 172.21.15.170:0/1223643684 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff8f01475d0 msgr2=0x7ff8f013cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:19.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.623+0000 7ff8f677a640 1 --2- 172.21.15.170:0/1223643684 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff8f01475d0 0x7ff8f013cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.623+0000 7ff8f677a640 1 -- 172.21.15.170:0/1223643684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8f01479b0 msgr2=0x7ff8f00b1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:19.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.623+0000 7ff8f677a640 1 --2- 172.21.15.170:0/1223643684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8f01479b0 0x7ff8f00b1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.623+0000 7ff8f677a640 1 -- 172.21.15.170:0/1223643684 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f00b0e20 con 0x7ff8f00b02d0 2024-05-08T04:13:19.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.623+0000 7ff8f5f79640 1 --2- 172.21.15.170:0/1223643684 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff8f01475d0 0x7ff8f013cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:13:19.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.624+0000 7ff8f677a640 1 --2- 172.21.15.170:0/1223643684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8f00b02d0 0x7ff8f013af60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff8e002b950 tx=0x7ff8e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:19.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.624+0000 7ff8ee7fc640 1 -- 172.21.15.170:0/1223643684 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0040070 con 0x7ff8f00b02d0 2024-05-08T04:13:19.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.624+0000 7ff8ee7fc640 1 -- 172.21.15.170:0/1223643684 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8e0030c50 con 0x7ff8f00b02d0 2024-05-08T04:13:19.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.624+0000 7ff8f81dd640 1 -- 172.21.15.170:0/1223643684 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f01d9050 con 0x7ff8f00b02d0 2024-05-08T04:13:19.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.624+0000 7ff8ee7fc640 1 -- 172.21.15.170:0/1223643684 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0035040 con 0x7ff8f00b02d0 2024-05-08T04:13:19.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.624+0000 7ff8f81dd640 1 -- 172.21.15.170:0/1223643684 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f013b680 con 0x7ff8f00b02d0 2024-05-08T04:13:19.628 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.625+0000 7ff8f81dd640 1 -- 172.21.15.170:0/1223643684 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8f006e9e0 con 0x7ff8f00b02d0 2024-05-08T04:13:19.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.625+0000 7ff8ee7fc640 1 -- 172.21.15.170:0/1223643684 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7ff8e0038020 con 0x7ff8f00b02d0 2024-05-08T04:13:19.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.626+0000 7ff8ee7fc640 1 --2- 172.21.15.170:0/1223643684 >> v2:172.21.15.187:6816/101007897 conn(0x7ff8dc0762c0 0x7ff8dc078780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.626+0000 7ff8ee7fc640 1 -- 172.21.15.170:0/1223643684 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 4525+0+0 (secure 0 0 0) 0x7ff8e00b8df0 con 0x7ff8f00b02d0 2024-05-08T04:13:19.629 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.626+0000 7ff8f5f79640 1 --2- 172.21.15.170:0/1223643684 >> v2:172.21.15.187:6816/101007897 conn(0x7ff8dc0762c0 0x7ff8dc078780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.630 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.626+0000 7ff8f5f79640 1 --2- 172.21.15.170:0/1223643684 >> v2:172.21.15.187:6816/101007897 conn(0x7ff8dc0762c0 0x7ff8dc078780 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8000f60 tx=0x7ff8e8007cf0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:19.631 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.630+0000 7ff8ee7fc640 1 -- 172.21.15.170:0/1223643684 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff8e0082630 con 0x7ff8f00b02d0 2024-05-08T04:13:19.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.855+0000 7ff8f81dd640 1 -- 172.21.15.170:0/1223643684 --> v2:172.21.15.187:6816/101007897 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7ff8f013b8b0 con 0x7ff8dc0762c0 2024-05-08T04:13:19.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.856+0000 7ff8ee7fc640 1 -- 172.21.15.170:0/1223643684 <== mgr.4107 v2:172.21.15.187:6816/101007897 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff8f013b8b0 con 0x7ff8dc0762c0 2024-05-08T04:13:19.857 INFO:teuthology.orchestra.run.smithi170.stderr:instructed osd(s) 5 to deep-scrub 2024-05-08T04:13:19.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.858+0000 7ff8f81dd640 1 -- 172.21.15.170:0/1223643684 >> v2:172.21.15.187:6816/101007897 conn(0x7ff8dc0762c0 msgr2=0x7ff8dc078780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:19.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.858+0000 7ff8f81dd640 1 --2- 172.21.15.170:0/1223643684 >> v2:172.21.15.187:6816/101007897 conn(0x7ff8dc0762c0 0x7ff8dc078780 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8000f60 tx=0x7ff8e8007cf0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.858+0000 7ff8f81dd640 1 -- 172.21.15.170:0/1223643684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8f00b02d0 msgr2=0x7ff8f013af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:19.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.858+0000 7ff8f81dd640 1 --2- 172.21.15.170:0/1223643684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8f00b02d0 0x7ff8f013af60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff8e002b950 tx=0x7ff8e00046c0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.858+0000 7ff8f81dd640 1 -- 172.21.15.170:0/1223643684 shutdown_connections 2024-05-08T04:13:19.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.858+0000 7ff8f81dd640 1 --2- 172.21.15.170:0/1223643684 >> v2:172.21.15.187:6816/101007897 conn(0x7ff8dc0762c0 0x7ff8dc078780 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.858+0000 7ff8f81dd640 1 --2- 172.21.15.170:0/1223643684 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff8f01479b0 0x7ff8f00b1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.858+0000 7ff8f81dd640 1 --2- 172.21.15.170:0/1223643684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8f00b02d0 0x7ff8f013af60 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.858+0000 7ff8f81dd640 1 --2- 172.21.15.170:0/1223643684 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff8f01475d0 0x7ff8f013cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.858+0000 7ff8f81dd640 1 -- 172.21.15.170:0/1223643684 >> 172.21.15.170:0/1223643684 conn(0x7ff8f00b9e60 msgr2=0x7ff8f013f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:19.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.858+0000 7ff8f81dd640 1 -- 172.21.15.170:0/1223643684 shutdown_connections 2024-05-08T04:13:19.861 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.858+0000 7ff8f81dd640 1 -- 172.21.15.170:0/1223643684 wait complete. 2024-05-08T04:13:19.873 INFO:tasks.ceph:Scrubbing osd.6 2024-05-08T04:13:19.874 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-08T04:13:19.982 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.981+0000 7f0c6f56e640 1 Processor -- start 2024-05-08T04:13:19.983 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.981+0000 7f0c6f56e640 1 -- start start 2024-05-08T04:13:19.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.981+0000 7f0c6f56e640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0c68147880 0x7f0c681433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.981+0000 7f0c6f56e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0c68143900 0x7f0c68070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.981+0000 7f0c6f56e640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c680b7470 con 0x7f0c680710f0 2024-05-08T04:13:19.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.981+0000 7f0c6f56e640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7f0c681484f0 con 0x7f0c68147880 2024-05-08T04:13:19.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.982+0000 7f0c6f56e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f0c68147fd0 con 0x7f0c68143900 2024-05-08T04:13:19.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.982+0000 7f0c6d30a640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f0c68147880 0x7f0c681433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.982+0000 7f0c6cb09640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0c68143900 0x7f0c68070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.982+0000 7f0c6db0b640 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f0c680710f0 0x7f0c68144090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43790/0 (socket says 172.21.15.170:43790) 2024-05-08T04:13:19.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.982+0000 7f0c6db0b640 1 -- 172.21.15.170:0/1479371774 learned_addr learned my addr 172.21.15.170:0/1479371774 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-05-08T04:13:19.984 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.982+0000 7f0c6d30a640 1 -- 172.21.15.170:0/1479371774 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0c68143900 msgr2=0x7f0c68070bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:13:19.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.982+0000 7f0c6d30a640 1 --2- 172.21.15.170:0/1479371774 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0c68143900 0x7f0c68070bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.982+0000 7f0c6d30a640 1 -- 172.21.15.170:0/1479371774 >> v1:172.21.15.170:6789/0 conn(0x7f0c680710f0 legacy=0x7f0c68144090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:19.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.982+0000 7f0c6d30a640 1 -- 172.21.15.170:0/1479371774 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c68147c60 con 0x7f0c68147880 2024-05-08T04:13:19.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.982+0000 7f0c5ffff640 1 -- 172.21.15.170:0/1479371774 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 688913022 0 0) 0x7f0c680b7470 con 0x7f0c680710f0 2024-05-08T04:13:19.985 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.982+0000 7f0c6cb09640 1 --2- 172.21.15.170:0/1479371774 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0c68143900 0x7f0c68070bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:13:19.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.983+0000 7f0c6d30a640 1 --2- 172.21.15.170:0/1479371774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c68147880 0x7f0c681433c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0c640097a0 tx=0x7f0c6402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc4805fd47efc5b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:19.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.983+0000 7f0c5ffff640 1 -- 172.21.15.170:0/1479371774 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c64031780 con 0x7f0c68147880 2024-05-08T04:13:19.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.983+0000 7f0c5ffff640 1 -- 172.21.15.170:0/1479371774 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c64037070 con 0x7f0c68147880 2024-05-08T04:13:19.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.983+0000 7f0c5ffff640 1 -- 172.21.15.170:0/1479371774 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c64031b40 con 0x7f0c68147880 2024-05-08T04:13:19.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.983+0000 7f0c6f56e640 1 -- 172.21.15.170:0/1479371774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c68147880 msgr2=0x7f0c681433c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:19.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.983+0000 7f0c6f56e640 1 --2- 172.21.15.170:0/1479371774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c68147880 0x7f0c681433c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0c640097a0 tx=0x7f0c6402b820 comp rx=0 tx=0).stop 2024-05-08T04:13:19.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.983+0000 7f0c6f56e640 1 -- 172.21.15.170:0/1479371774 shutdown_connections 2024-05-08T04:13:19.988 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.983+0000 7f0c6f56e640 1 --2- 172.21.15.170:0/1479371774 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0c68143900 0x7f0c68070bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.983+0000 7f0c6f56e640 1 --2- 172.21.15.170:0/1479371774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c68147880 0x7f0c681433c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.983+0000 7f0c6f56e640 1 -- 172.21.15.170:0/1479371774 >> 172.21.15.170:0/1479371774 conn(0x7f0c680b6570 msgr2=0x7f0c680b6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:19.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.984+0000 7f0c6f56e640 1 -- 172.21.15.170:0/1479371774 shutdown_connections 2024-05-08T04:13:19.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.984+0000 7f0c6f56e640 1 -- 172.21.15.170:0/1479371774 wait complete. 2024-05-08T04:13:19.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.984+0000 7f0c6f56e640 1 Processor -- start 2024-05-08T04:13:19.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.985+0000 7f0c6f56e640 1 -- start start 2024-05-08T04:13:19.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.985+0000 7f0c6f56e640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0c680710f0 0x7f0c6813a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.985+0000 7f0c6f56e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c68143900 0x7f0c680af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.985+0000 7f0c6f56e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c68147880 0x7f0c681d9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.985+0000 7f0c6f56e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0c68074100 con 0x7f0c68143900 2024-05-08T04:13:19.989 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.985+0000 7f0c6f56e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0c68073f80 con 0x7f0c68147880 2024-05-08T04:13:19.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.985+0000 7f0c6f56e640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f0c68071520 con 0x7f0c680710f0 2024-05-08T04:13:19.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.985+0000 7f0c6cb09640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c68143900 0x7f0c680af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.985+0000 7f0c6d30a640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0c680710f0 0x7f0c6813a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.985+0000 7f0c6cb09640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c68143900 0x7f0c680af7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49566/0 (socket says 172.21.15.170:49566) 2024-05-08T04:13:19.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.985+0000 7f0c6cb09640 1 -- 172.21.15.170:0/298629043 learned_addr learned my addr 172.21.15.170:0/298629043 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:13:19.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.985+0000 7f0c6db0b640 1 --2- 172.21.15.170:0/298629043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c68147880 0x7f0c681d9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.985+0000 7f0c6d30a640 1 -- 172.21.15.170:0/298629043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c68147880 msgr2=0x7f0c681d9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:19.990 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.985+0000 7f0c6d30a640 1 --2- 172.21.15.170:0/298629043 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0c68147880 0x7f0c681d9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.985+0000 7f0c6d30a640 1 -- 172.21.15.170:0/298629043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c68143900 msgr2=0x7f0c680af7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:19.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.985+0000 7f0c6d30a640 1 --2- 172.21.15.170:0/298629043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c68143900 0x7f0c680af7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:19.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.985+0000 7f0c6d30a640 1 -- 172.21.15.170:0/298629043 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c6813aeb0 con 0x7f0c680710f0 2024-05-08T04:13:19.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.985+0000 7f0c6cb09640 1 --2- 172.21.15.170:0/298629043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c68143900 0x7f0c680af7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:13:19.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.986+0000 7f0c6d30a640 1 --2- 172.21.15.170:0/298629043 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0c680710f0 0x7f0c6813a970 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0c64033000 tx=0x7f0c640046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:19.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.986+0000 7f0c5dffb640 1 -- 172.21.15.170:0/298629043 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c64032610 con 0x7f0c680710f0 2024-05-08T04:13:19.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.986+0000 7f0c5dffb640 1 -- 172.21.15.170:0/298629043 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c640327d0 con 0x7f0c680710f0 2024-05-08T04:13:19.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.986+0000 7f0c6f56e640 1 -- 172.21.15.170:0/298629043 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c680b1cb0 con 0x7f0c680710f0 2024-05-08T04:13:19.991 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.986+0000 7f0c5dffb640 1 -- 172.21.15.170:0/298629043 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c64032bc0 con 0x7f0c680710f0 2024-05-08T04:13:19.992 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.986+0000 7f0c6f56e640 1 -- 172.21.15.170:0/298629043 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c680afd30 con 0x7f0c680710f0 2024-05-08T04:13:19.993 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.987+0000 7f0c6f56e640 1 -- 172.21.15.170:0/298629043 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0c681d9ab0 con 0x7f0c680710f0 2024-05-08T04:13:19.993 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.988+0000 7f0c5dffb640 1 -- 172.21.15.170:0/298629043 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f0c6404e440 con 0x7f0c680710f0 2024-05-08T04:13:19.993 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.988+0000 7f0c5dffb640 1 --2- 172.21.15.170:0/298629043 >> v2:172.21.15.187:6816/101007897 conn(0x7f0c4c076290 0x7f0c4c078750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.993 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.988+0000 7f0c5dffb640 1 -- 172.21.15.170:0/298629043 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 4525+0+0 (secure 0 0 0) 0x7f0c640b8bd0 con 0x7f0c680710f0 2024-05-08T04:13:19.993 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.988+0000 7f0c6cb09640 1 --2- 172.21.15.170:0/298629043 >> v2:172.21.15.187:6816/101007897 conn(0x7f0c4c076290 0x7f0c4c078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.993 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.988+0000 7f0c5dffb640 1 --2- 172.21.15.170:0/298629043 >> v2:172.21.15.187:6809/2911331731 conn(0x7f0c4c07cff0 0x7f0c4c07f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:19.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.988+0000 7f0c5dffb640 1 -- 172.21.15.170:0/298629043 --> v2:172.21.15.187:6809/2911331731 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0c64033ec0 con 0x7f0c4c07cff0 2024-05-08T04:13:19.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.988+0000 7f0c5dffb640 1 -- 172.21.15.170:0/298629043 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f0c640be2a0 con 0x7f0c680710f0 2024-05-08T04:13:19.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.988+0000 7f0c6db0b640 1 --2- 172.21.15.170:0/298629043 >> v2:172.21.15.187:6809/2911331731 conn(0x7f0c4c07cff0 0x7f0c4c07f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:19.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.989+0000 7f0c6cb09640 1 --2- 172.21.15.170:0/298629043 >> v2:172.21.15.187:6816/101007897 conn(0x7f0c4c076290 0x7f0c4c078750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0c680b1900 tx=0x7f0c58009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:19.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.989+0000 7f0c6db0b640 1 --2- 172.21.15.170:0/298629043 >> v2:172.21.15.187:6809/2911331731 conn(0x7f0c4c07cff0 0x7f0c4c07f410 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:19.994 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:19.990+0000 7f0c5dffb640 1 -- 172.21.15.170:0/298629043 <== osd.6 v2:172.21.15.187:6809/2911331731 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0c64033ec0 con 0x7f0c4c07cff0 2024-05-08T04:13:20.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.002+0000 7f0c6f56e640 1 -- 172.21.15.170:0/298629043 --> v2:172.21.15.187:6809/2911331731 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0c6806d400 con 0x7f0c4c07cff0 2024-05-08T04:13:20.004 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.003+0000 7f0c5dffb640 1 -- 172.21.15.170:0/298629043 <== osd.6 v2:172.21.15.187:6809/2911331731 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0c6806d400 con 0x7f0c4c07cff0 2024-05-08T04:13:20.005 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-05-08T04:13:20.005 INFO:teuthology.orchestra.run.smithi170.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T04:13:20.005 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-05-08T04:13:20.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.003+0000 7f0c6f56e640 1 -- 172.21.15.170:0/298629043 >> v2:172.21.15.187:6809/2911331731 conn(0x7f0c4c07cff0 msgr2=0x7f0c4c07f410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:20.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.003+0000 7f0c6f56e640 1 --2- 172.21.15.170:0/298629043 >> v2:172.21.15.187:6809/2911331731 conn(0x7f0c4c07cff0 0x7f0c4c07f410 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:20.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.003+0000 7f0c6f56e640 1 -- 172.21.15.170:0/298629043 >> v2:172.21.15.187:6816/101007897 conn(0x7f0c4c076290 msgr2=0x7f0c4c078750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:20.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.003+0000 7f0c6f56e640 1 --2- 172.21.15.170:0/298629043 >> v2:172.21.15.187:6816/101007897 conn(0x7f0c4c076290 0x7f0c4c078750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0c680b1900 tx=0x7f0c58009190 comp rx=0 tx=0).stop 2024-05-08T04:13:20.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.003+0000 7f0c6f56e640 1 -- 172.21.15.170:0/298629043 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0c680710f0 msgr2=0x7f0c6813a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:20.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.003+0000 7f0c6f56e640 1 --2- 172.21.15.170:0/298629043 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f0c680710f0 0x7f0c6813a970 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0c64033000 tx=0x7f0c640046c0 comp rx=0 tx=0).stop 2024-05-08T04:13:20.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.003+0000 7f0c6db0b640 1 -- 172.21.15.170:0/298629043 reap_dead start 2024-05-08T04:13:20.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.003+0000 7f0c6f56e640 1 -- 172.21.15.170:0/298629043 shutdown_connections 2024-05-08T04:13:20.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.003+0000 7f0c6f56e640 1 -- 172.21.15.170:0/298629043 >> 172.21.15.170:0/298629043 conn(0x7f0c680b6570 msgr2=0x7f0c68072b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:20.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.003+0000 7f0c6f56e640 1 -- 172.21.15.170:0/298629043 shutdown_connections 2024-05-08T04:13:20.006 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.003+0000 7f0c6f56e640 1 -- 172.21.15.170:0/298629043 wait complete. 2024-05-08T04:13:20.016 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-08T04:13:20.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.124+0000 7fbcd5283640 1 Processor -- start 2024-05-08T04:13:20.126 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.124+0000 7fbcd5283640 1 -- start start 2024-05-08T04:13:20.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.125+0000 7fbcd5283640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fbcd0143ea0 0x7fbcd0072bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:20.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.125+0000 7fbcd5283640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbcd0143320 0x7fbcd0143790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:20.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.125+0000 7fbcd5283640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcd00b7230 con 0x7fbcd00b6e50 2024-05-08T04:13:20.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.125+0000 7fbcd5283640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7fbcd01442e0 con 0x7fbcd0143ea0 2024-05-08T04:13:20.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.125+0000 7fbcd5283640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fbcd0074950 con 0x7fbcd0143320 2024-05-08T04:13:20.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.125+0000 7fbcce7fc640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fbcd0143ea0 0x7fbcd0072bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:20.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.125+0000 7fbccf7fe640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbcd0143320 0x7fbcd0143790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:20.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.125+0000 7fbcceffd640 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fbcd00b6e50 0x7fbcd0147ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43806/0 (socket says 172.21.15.170:43806) 2024-05-08T04:13:20.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.125+0000 7fbccf7fe640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbcd0143320 0x7fbcd0143790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:48168/0 (socket says 172.21.15.170:48168) 2024-05-08T04:13:20.128 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.125+0000 7fbcce7fc640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fbcd0143ea0 0x7fbcd0072bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49582/0 (socket says 172.21.15.170:49582) 2024-05-08T04:13:20.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.125+0000 7fbcceffd640 1 -- 172.21.15.170:0/3456636212 learned_addr learned my addr 172.21.15.170:0/3456636212 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-05-08T04:13:20.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.125+0000 7fbccdffb640 1 -- 172.21.15.170:0/3456636212 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3095649432 0 0) 0x7fbcd00b7230 con 0x7fbcd00b6e50 2024-05-08T04:13:20.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.125+0000 7fbccdffb640 1 -- 172.21.15.170:0/3456636212 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbcb8003520 con 0x7fbcd00b6e50 2024-05-08T04:13:20.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.125+0000 7fbccf7fe640 1 -- 172.21.15.170:0/3456636212 >> v2:172.21.15.170:3300/0 conn(0x7fbcd0143ea0 msgr2=0x7fbcd0072bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:13:20.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.125+0000 7fbccf7fe640 1 --2- 172.21.15.170:0/3456636212 >> v2:172.21.15.170:3300/0 conn(0x7fbcd0143ea0 0x7fbcd0072bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:20.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.125+0000 7fbccf7fe640 1 -- 172.21.15.170:0/3456636212 >> v1:172.21.15.170:6789/0 conn(0x7fbcd00b6e50 legacy=0x7fbcd0147ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:20.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.125+0000 7fbccf7fe640 1 -- 172.21.15.170:0/3456636212 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd01475c0 con 0x7fbcd0143320 2024-05-08T04:13:20.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.126+0000 7fbccf7fe640 1 --2- 172.21.15.170:0/3456636212 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbcd0143320 0x7fbcd0143790 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8009d90 tx=0x7fbcc802be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3922a37cbdacf70a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:20.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.126+0000 7fbccdffb640 1 -- 172.21.15.170:0/3456636212 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8039070 con 0x7fbcd0143320 2024-05-08T04:13:20.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.126+0000 7fbccdffb640 1 -- 172.21.15.170:0/3456636212 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcc802cbd0 con 0x7fbcd0143320 2024-05-08T04:13:20.129 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.126+0000 7fbccdffb640 1 -- 172.21.15.170:0/3456636212 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8034440 con 0x7fbcd0143320 2024-05-08T04:13:20.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.127+0000 7fbcd5283640 1 -- 172.21.15.170:0/3456636212 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbcd0143320 msgr2=0x7fbcd0143790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:20.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.127+0000 7fbcd5283640 1 --2- 172.21.15.170:0/3456636212 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbcd0143320 0x7fbcd0143790 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8009d90 tx=0x7fbcc802be20 comp rx=0 tx=0).stop 2024-05-08T04:13:20.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.127+0000 7fbcd5283640 1 -- 172.21.15.170:0/3456636212 shutdown_connections 2024-05-08T04:13:20.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.127+0000 7fbcd5283640 1 --2- 172.21.15.170:0/3456636212 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbcd0143320 0x7fbcd0143790 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:20.130 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.127+0000 7fbcd5283640 1 --2- 172.21.15.170:0/3456636212 >> v2:172.21.15.170:3300/0 conn(0x7fbcd0143ea0 0x7fbcd0072bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:20.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.127+0000 7fbcd5283640 1 -- 172.21.15.170:0/3456636212 >> 172.21.15.170:0/3456636212 conn(0x7fbcd00ba000 msgr2=0x7fbcd00b64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:20.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.127+0000 7fbcd5283640 1 -- 172.21.15.170:0/3456636212 shutdown_connections 2024-05-08T04:13:20.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.127+0000 7fbcd5283640 1 -- 172.21.15.170:0/3456636212 wait complete. 2024-05-08T04:13:20.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.128+0000 7fbcd5283640 1 Processor -- start 2024-05-08T04:13:20.131 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.129+0000 7fbcd5283640 1 -- start start 2024-05-08T04:13:20.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.129+0000 7fbcd5283640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbcd00b6e50 0x7fbcd013d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:20.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.129+0000 7fbcd5283640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbcd0143320 0x7fbcd0137c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:20.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.129+0000 7fbcd5283640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcd0143ea0 0x7fbcd01381a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:20.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.129+0000 7fbcd5283640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd0073eb0 con 0x7fbcd00b6e50 2024-05-08T04:13:20.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.129+0000 7fbcd5283640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd0073d30 con 0x7fbcd0143ea0 2024-05-08T04:13:20.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.129+0000 7fbcd5283640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fbcd014e7b0 con 0x7fbcd0143320 2024-05-08T04:13:20.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.129+0000 7fbcceffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbcd00b6e50 0x7fbcd013d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:20.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.129+0000 7fbcce7fc640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbcd0143320 0x7fbcd0137c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:20.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.129+0000 7fbcce7fc640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbcd0143320 0x7fbcd0137c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:48180/0 (socket says 172.21.15.170:48180) 2024-05-08T04:13:20.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.129+0000 7fbcce7fc640 1 -- 172.21.15.170:0/1893714438 learned_addr learned my addr 172.21.15.170:0/1893714438 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:13:20.133 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.129+0000 7fbccf7fe640 1 --2- 172.21.15.170:0/1893714438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcd0143ea0 0x7fbcd01381a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:20.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.130+0000 7fbcce7fc640 1 -- 172.21.15.170:0/1893714438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcd0143ea0 msgr2=0x7fbcd01381a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:20.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.130+0000 7fbcce7fc640 1 --2- 172.21.15.170:0/1893714438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcd0143ea0 0x7fbcd01381a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:20.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.130+0000 7fbcce7fc640 1 -- 172.21.15.170:0/1893714438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbcd00b6e50 msgr2=0x7fbcd013d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:20.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.130+0000 7fbcce7fc640 1 --2- 172.21.15.170:0/1893714438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbcd00b6e50 0x7fbcd013d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:20.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.130+0000 7fbcce7fc640 1 -- 172.21.15.170:0/1893714438 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd013dfe0 con 0x7fbcd0143320 2024-05-08T04:13:20.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.130+0000 7fbcceffd640 1 --2- 172.21.15.170:0/1893714438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbcd00b6e50 0x7fbcd013d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:13:20.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.130+0000 7fbcce7fc640 1 --2- 172.21.15.170:0/1893714438 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbcd0143320 0x7fbcd0137c60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbcc400e870 tx=0x7fbcc400ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:20.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.130+0000 7fbcb77fe640 1 -- 172.21.15.170:0/1893714438 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4017070 con 0x7fbcd0143320 2024-05-08T04:13:20.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.130+0000 7fbcb77fe640 1 -- 172.21.15.170:0/1893714438 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcc400c940 con 0x7fbcd0143320 2024-05-08T04:13:20.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.130+0000 7fbcd5283640 1 -- 172.21.15.170:0/1893714438 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcd01487c0 con 0x7fbcd0143320 2024-05-08T04:13:20.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.130+0000 7fbcb77fe640 1 -- 172.21.15.170:0/1893714438 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc400cd30 con 0x7fbcd0143320 2024-05-08T04:13:20.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.130+0000 7fbcd5283640 1 -- 172.21.15.170:0/1893714438 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcd014e2d0 con 0x7fbcd0143320 2024-05-08T04:13:20.134 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.131+0000 7fbcd5283640 1 -- 172.21.15.170:0/1893714438 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcd006e9e0 con 0x7fbcd0143320 2024-05-08T04:13:20.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.132+0000 7fbcb77fe640 1 -- 172.21.15.170:0/1893714438 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7fbcc401a070 con 0x7fbcd0143320 2024-05-08T04:13:20.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.132+0000 7fbcb77fe640 1 --2- 172.21.15.170:0/1893714438 >> v2:172.21.15.187:6816/101007897 conn(0x7fbcbc076290 0x7fbcbc078750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:20.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.133+0000 7fbcb77fe640 1 -- 172.21.15.170:0/1893714438 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 4525+0+0 (secure 0 0 0) 0x7fbcc4096f10 con 0x7fbcd0143320 2024-05-08T04:13:20.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.133+0000 7fbcceffd640 1 --2- 172.21.15.170:0/1893714438 >> v2:172.21.15.187:6816/101007897 conn(0x7fbcbc076290 0x7fbcbc078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:20.135 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.133+0000 7fbcceffd640 1 --2- 172.21.15.170:0/1893714438 >> v2:172.21.15.187:6816/101007897 conn(0x7fbcbc076290 0x7fbcbc078750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0007a60 tx=0x7fbcc00079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:20.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.139+0000 7fbcb77fe640 1 -- 172.21.15.170:0/1893714438 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbcc40606c0 con 0x7fbcd0143320 2024-05-08T04:13:20.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.400+0000 7fbcd5283640 1 -- 172.21.15.170:0/1893714438 --> v2:172.21.15.187:6816/101007897 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fbcd006b400 con 0x7fbcbc076290 2024-05-08T04:13:20.402 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.401+0000 7fbcb77fe640 1 -- 172.21.15.170:0/1893714438 <== mgr.4107 v2:172.21.15.187:6816/101007897 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fbcd006b400 con 0x7fbcbc076290 2024-05-08T04:13:20.403 INFO:teuthology.orchestra.run.smithi170.stderr:instructed osd(s) 6 to deep-scrub 2024-05-08T04:13:20.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.404+0000 7fbcd5283640 1 -- 172.21.15.170:0/1893714438 >> v2:172.21.15.187:6816/101007897 conn(0x7fbcbc076290 msgr2=0x7fbcbc078750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:20.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.404+0000 7fbcd5283640 1 --2- 172.21.15.170:0/1893714438 >> v2:172.21.15.187:6816/101007897 conn(0x7fbcbc076290 0x7fbcbc078750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0007a60 tx=0x7fbcc00079b0 comp rx=0 tx=0).stop 2024-05-08T04:13:20.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.404+0000 7fbcd5283640 1 -- 172.21.15.170:0/1893714438 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbcd0143320 msgr2=0x7fbcd0137c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:20.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.404+0000 7fbcd5283640 1 --2- 172.21.15.170:0/1893714438 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbcd0143320 0x7fbcd0137c60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbcc400e870 tx=0x7fbcc400ed30 comp rx=0 tx=0).stop 2024-05-08T04:13:20.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.404+0000 7fbcd5283640 1 -- 172.21.15.170:0/1893714438 shutdown_connections 2024-05-08T04:13:20.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.404+0000 7fbcd5283640 1 --2- 172.21.15.170:0/1893714438 >> v2:172.21.15.187:6816/101007897 conn(0x7fbcbc076290 0x7fbcbc078750 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:20.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.404+0000 7fbcd5283640 1 --2- 172.21.15.170:0/1893714438 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbcd0143ea0 0x7fbcd01381a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:20.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.404+0000 7fbcd5283640 1 --2- 172.21.15.170:0/1893714438 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fbcd0143320 0x7fbcd0137c60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:20.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.404+0000 7fbcd5283640 1 --2- 172.21.15.170:0/1893714438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbcd00b6e50 0x7fbcd013d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:20.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.404+0000 7fbcd5283640 1 -- 172.21.15.170:0/1893714438 >> 172.21.15.170:0/1893714438 conn(0x7fbcd00ba000 msgr2=0x7fbcd006faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:20.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.404+0000 7fbcd5283640 1 -- 172.21.15.170:0/1893714438 shutdown_connections 2024-05-08T04:13:20.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.404+0000 7fbcd5283640 1 -- 172.21.15.170:0/1893714438 wait complete. 2024-05-08T04:13:20.419 INFO:tasks.ceph:Scrubbing osd.7 2024-05-08T04:13:20.420 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-08T04:13:20.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.537+0000 7fb8977ab640 1 Processor -- start 2024-05-08T04:13:20.539 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.537+0000 7fb8977ab640 1 -- start start 2024-05-08T04:13:20.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.538+0000 7fb8977ab640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb890143ad0 0x7fb890070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:20.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.538+0000 7fb8977ab640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb8900710f0 0x7fb89014e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:20.540 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.538+0000 7fb8977ab640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8900b7470 con 0x7fb890147880 2024-05-08T04:13:20.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.538+0000 7fb8977ab640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb8901484f0 con 0x7fb890143ad0 2024-05-08T04:13:20.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.538+0000 7fb8977ab640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fb890147fd0 con 0x7fb8900710f0 2024-05-08T04:13:20.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.538+0000 7fb895d48640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb8900710f0 0x7fb89014e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:20.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.538+0000 7fb895d48640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb8900710f0 0x7fb89014e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:48186/0 (socket says 172.21.15.170:48186) 2024-05-08T04:13:20.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.538+0000 7fb895d48640 1 -- 172.21.15.170:0/477702068 learned_addr learned my addr 172.21.15.170:0/477702068 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:13:20.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.538+0000 7fb894d46640 1 --2- 172.21.15.170:0/477702068 >> v2:172.21.15.187:3300/0 conn(0x7fb890143ad0 0x7fb890070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:20.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.539+0000 7fb895d48640 1 -- 172.21.15.170:0/477702068 >> v2:172.21.15.187:3300/0 conn(0x7fb890143ad0 msgr2=0x7fb890070bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:13:20.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.539+0000 7fb895d48640 1 --2- 172.21.15.170:0/477702068 >> v2:172.21.15.187:3300/0 conn(0x7fb890143ad0 0x7fb890070bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:20.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.539+0000 7fb895d48640 1 -- 172.21.15.170:0/477702068 >> v1:172.21.15.187:6789/0 conn(0x7fb890147880 legacy=0x7fb8901433c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:13:20.541 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.539+0000 7fb895d48640 1 -- 172.21.15.170:0/477702068 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb890147c60 con 0x7fb8900710f0 2024-05-08T04:13:20.542 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.539+0000 7fb895d48640 1 --2- 172.21.15.170:0/477702068 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb8900710f0 0x7fb89014e9a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb888009e50 tx=0x7fb88802be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ffab5df298a63f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:20.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.540+0000 7fb887fff640 1 -- 172.21.15.170:0/477702068 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88802c9e0 con 0x7fb8900710f0 2024-05-08T04:13:20.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.540+0000 7fb887fff640 1 -- 172.21.15.170:0/477702068 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb88802cb80 con 0x7fb8900710f0 2024-05-08T04:13:20.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.540+0000 7fb887fff640 1 -- 172.21.15.170:0/477702068 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8880343f0 con 0x7fb8900710f0 2024-05-08T04:13:20.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.540+0000 7fb8977ab640 1 -- 172.21.15.170:0/477702068 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb8900710f0 msgr2=0x7fb89014e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:20.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.540+0000 7fb8977ab640 1 --2- 172.21.15.170:0/477702068 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb8900710f0 0x7fb89014e9a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb888009e50 tx=0x7fb88802be40 comp rx=0 tx=0).stop 2024-05-08T04:13:20.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.540+0000 7fb8977ab640 1 -- 172.21.15.170:0/477702068 shutdown_connections 2024-05-08T04:13:20.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.540+0000 7fb8977ab640 1 --2- 172.21.15.170:0/477702068 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb8900710f0 0x7fb89014e9a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:20.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.540+0000 7fb8977ab640 1 --2- 172.21.15.170:0/477702068 >> v2:172.21.15.187:3300/0 conn(0x7fb890143ad0 0x7fb890070bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:20.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.540+0000 7fb8977ab640 1 -- 172.21.15.170:0/477702068 >> 172.21.15.170:0/477702068 conn(0x7fb8900b6570 msgr2=0x7fb8900b6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:20.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.541+0000 7fb8977ab640 1 -- 172.21.15.170:0/477702068 shutdown_connections 2024-05-08T04:13:20.543 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.541+0000 7fb8977ab640 1 -- 172.21.15.170:0/477702068 wait complete. 2024-05-08T04:13:20.545 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.542+0000 7fb8977ab640 1 Processor -- start 2024-05-08T04:13:20.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.542+0000 7fb8977ab640 1 -- start start 2024-05-08T04:13:20.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.542+0000 7fb8977ab640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8900710f0 0x7fb8901395a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:20.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.542+0000 7fb8977ab640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb890143ad0 0x7fb890138a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:20.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.542+0000 7fb8977ab640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb890147880 0x7fb89013b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:20.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.542+0000 7fb8977ab640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb890072a90 con 0x7fb890143ad0 2024-05-08T04:13:20.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.542+0000 7fb8977ab640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb890072910 con 0x7fb8900710f0 2024-05-08T04:13:20.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.542+0000 7fb8977ab640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fb890073f80 con 0x7fb890147880 2024-05-08T04:13:20.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.542+0000 7fb895d48640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb890147880 0x7fb89013b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:20.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.542+0000 7fb894d46640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb890143ad0 0x7fb890138a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:20.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.542+0000 7fb895d48640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb890147880 0x7fb89013b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:48202/0 (socket says 172.21.15.170:48202) 2024-05-08T04:13:20.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.542+0000 7fb895d48640 1 -- 172.21.15.170:0/1603671711 learned_addr learned my addr 172.21.15.170:0/1603671711 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:13:20.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.542+0000 7fb895547640 1 --2- 172.21.15.170:0/1603671711 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8900710f0 0x7fb8901395a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:20.546 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.543+0000 7fb894d46640 1 -- 172.21.15.170:0/1603671711 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb890147880 msgr2=0x7fb89013b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:20.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.543+0000 7fb894d46640 1 --2- 172.21.15.170:0/1603671711 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb890147880 0x7fb89013b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:20.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.543+0000 7fb894d46640 1 -- 172.21.15.170:0/1603671711 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8900710f0 msgr2=0x7fb8901395a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:20.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.543+0000 7fb894d46640 1 --2- 172.21.15.170:0/1603671711 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8900710f0 0x7fb8901395a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:20.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.543+0000 7fb894d46640 1 -- 172.21.15.170:0/1603671711 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb89013a810 con 0x7fb890143ad0 2024-05-08T04:13:20.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.543+0000 7fb895d48640 1 --2- 172.21.15.170:0/1603671711 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fb890147880 0x7fb89013b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:13:20.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.543+0000 7fb895547640 1 --2- 172.21.15.170:0/1603671711 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb8900710f0 0x7fb8901395a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:13:20.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.543+0000 7fb894d46640 1 --2- 172.21.15.170:0/1603671711 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb890143ad0 0x7fb890138a80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb88000d5f0 tx=0x7fb88000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:20.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.543+0000 7fb885ffb640 1 -- 172.21.15.170:0/1603671711 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8800125f0 con 0x7fb890143ad0 2024-05-08T04:13:20.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.543+0000 7fb885ffb640 1 -- 172.21.15.170:0/1603671711 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb880019070 con 0x7fb890143ad0 2024-05-08T04:13:20.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.543+0000 7fb8977ab640 1 -- 172.21.15.170:0/1603671711 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb890138560 con 0x7fb890143ad0 2024-05-08T04:13:20.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.543+0000 7fb885ffb640 1 -- 172.21.15.170:0/1603671711 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8800129d0 con 0x7fb890143ad0 2024-05-08T04:13:20.547 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.544+0000 7fb8977ab640 1 -- 172.21.15.170:0/1603671711 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb89013b8b0 con 0x7fb890143ad0 2024-05-08T04:13:20.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.545+0000 7fb8977ab640 1 -- 172.21.15.170:0/1603671711 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb89006d400 con 0x7fb890143ad0 2024-05-08T04:13:20.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.545+0000 7fb885ffb640 1 -- 172.21.15.170:0/1603671711 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7fb880012b70 con 0x7fb890143ad0 2024-05-08T04:13:20.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.545+0000 7fb885ffb640 1 --2- 172.21.15.170:0/1603671711 >> v2:172.21.15.187:6816/101007897 conn(0x7fb874076290 0x7fb874078750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:20.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.545+0000 7fb885ffb640 1 -- 172.21.15.170:0/1603671711 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 4525+0+0 (secure 0 0 0) 0x7fb8800973b0 con 0x7fb890143ad0 2024-05-08T04:13:20.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.546+0000 7fb885ffb640 1 --2- 172.21.15.170:0/1603671711 >> v2:172.21.15.187:6804/2815417961 conn(0x7fb87407cff0 0x7fb87407f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:20.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.546+0000 7fb885ffb640 1 -- 172.21.15.170:0/1603671711 --> v2:172.21.15.187:6804/2815417961 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb880013e50 con 0x7fb87407cff0 2024-05-08T04:13:20.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.546+0000 7fb885ffb640 1 -- 172.21.15.170:0/1603671711 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb88009e050 con 0x7fb890143ad0 2024-05-08T04:13:20.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.546+0000 7fb895547640 1 --2- 172.21.15.170:0/1603671711 >> v2:172.21.15.187:6816/101007897 conn(0x7fb874076290 0x7fb874078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:20.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.546+0000 7fb895d48640 1 --2- 172.21.15.170:0/1603671711 >> v2:172.21.15.187:6804/2815417961 conn(0x7fb87407cff0 0x7fb87407f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:20.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.546+0000 7fb895d48640 1 --2- 172.21.15.170:0/1603671711 >> v2:172.21.15.187:6804/2815417961 conn(0x7fb87407cff0 0x7fb87407f410 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:20.549 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.546+0000 7fb895547640 1 --2- 172.21.15.170:0/1603671711 >> v2:172.21.15.187:6816/101007897 conn(0x7fb874076290 0x7fb874078750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb88c006fd0 tx=0x7fb88c009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:20.550 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.547+0000 7fb885ffb640 1 -- 172.21.15.170:0/1603671711 <== osd.7 v2:172.21.15.187:6804/2815417961 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb880013e50 con 0x7fb87407cff0 2024-05-08T04:13:20.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.560+0000 7fb8977ab640 1 -- 172.21.15.170:0/1603671711 --> v2:172.21.15.187:6804/2815417961 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb890136d10 con 0x7fb87407cff0 2024-05-08T04:13:20.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.561+0000 7fb885ffb640 1 -- 172.21.15.170:0/1603671711 <== osd.7 v2:172.21.15.187:6804/2815417961 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb890136d10 con 0x7fb87407cff0 2024-05-08T04:13:20.563 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-05-08T04:13:20.563 INFO:teuthology.orchestra.run.smithi170.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T04:13:20.563 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-05-08T04:13:20.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.561+0000 7fb8977ab640 1 -- 172.21.15.170:0/1603671711 >> v2:172.21.15.187:6804/2815417961 conn(0x7fb87407cff0 msgr2=0x7fb87407f410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:20.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.561+0000 7fb8977ab640 1 --2- 172.21.15.170:0/1603671711 >> v2:172.21.15.187:6804/2815417961 conn(0x7fb87407cff0 0x7fb87407f410 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:20.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.561+0000 7fb8977ab640 1 -- 172.21.15.170:0/1603671711 >> v2:172.21.15.187:6816/101007897 conn(0x7fb874076290 msgr2=0x7fb874078750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:20.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.561+0000 7fb8977ab640 1 --2- 172.21.15.170:0/1603671711 >> v2:172.21.15.187:6816/101007897 conn(0x7fb874076290 0x7fb874078750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb88c006fd0 tx=0x7fb88c009060 comp rx=0 tx=0).stop 2024-05-08T04:13:20.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.561+0000 7fb8977ab640 1 -- 172.21.15.170:0/1603671711 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb890143ad0 msgr2=0x7fb890138a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:20.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.561+0000 7fb8977ab640 1 --2- 172.21.15.170:0/1603671711 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb890143ad0 0x7fb890138a80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb88000d5f0 tx=0x7fb88000dab0 comp rx=0 tx=0).stop 2024-05-08T04:13:20.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.561+0000 7fb895d48640 1 -- 172.21.15.170:0/1603671711 reap_dead start 2024-05-08T04:13:20.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.561+0000 7fb8977ab640 1 -- 172.21.15.170:0/1603671711 shutdown_connections 2024-05-08T04:13:20.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.561+0000 7fb8977ab640 1 -- 172.21.15.170:0/1603671711 >> 172.21.15.170:0/1603671711 conn(0x7fb8900b6570 msgr2=0x7fb890143190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:20.564 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.561+0000 7fb8977ab640 1 -- 172.21.15.170:0/1603671711 shutdown_connections 2024-05-08T04:13:20.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.561+0000 7fb8977ab640 1 -- 172.21.15.170:0/1603671711 wait complete. 2024-05-08T04:13:20.574 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-08T04:13:20.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.675+0000 7fcc72f96640 1 Processor -- start 2024-05-08T04:13:20.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.675+0000 7fcc72f96640 1 -- start start 2024-05-08T04:13:20.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.676+0000 7fcc72f96640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc6c0b4d10 0x7fcc6c0aae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:20.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.676+0000 7fcc72f96640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fcc6c13bcc0 0x7fcc6c1459a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:20.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.676+0000 7fcc72f96640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc6c0b50f0 con 0x7fcc6c13b860 2024-05-08T04:13:20.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.676+0000 7fcc72f96640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fcc6c0acac0 con 0x7fcc6c13bcc0 2024-05-08T04:13:20.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.676+0000 7fcc72f96640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fcc6c0ab4d0 con 0x7fcc6c0b4d10 2024-05-08T04:13:20.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.676+0000 7fcc70d32640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc6c0b4d10 0x7fcc6c0aae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:20.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.676+0000 7fcc63fff640 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fcc6c13b860 0x7fcc6c145290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43818/0 (socket says 172.21.15.170:43818) 2024-05-08T04:13:20.679 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.676+0000 7fcc63fff640 1 -- 172.21.15.170:0/396104874 learned_addr learned my addr 172.21.15.170:0/396104874 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-05-08T04:13:20.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.676+0000 7fcc71533640 1 --2- 172.21.15.170:0/396104874 >> v2:172.21.15.187:3300/0 conn(0x7fcc6c13bcc0 0x7fcc6c1459a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:20.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.676+0000 7fcc70d32640 1 -- 172.21.15.170:0/396104874 >> v2:172.21.15.187:3300/0 conn(0x7fcc6c13bcc0 msgr2=0x7fcc6c1459a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:13:20.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.676+0000 7fcc70d32640 1 --2- 172.21.15.170:0/396104874 >> v2:172.21.15.187:3300/0 conn(0x7fcc6c13bcc0 0x7fcc6c1459a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:20.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.676+0000 7fcc70d32640 1 -- 172.21.15.170:0/396104874 >> v1:172.21.15.170:6789/0 conn(0x7fcc6c13b860 legacy=0x7fcc6c145290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:20.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.676+0000 7fcc637fe640 1 -- 172.21.15.170:0/396104874 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 297293479 0 0) 0x7fcc6c0b50f0 con 0x7fcc6c13b860 2024-05-08T04:13:20.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.676+0000 7fcc70d32640 1 -- 172.21.15.170:0/396104874 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc6c0ab870 con 0x7fcc6c0b4d10 2024-05-08T04:13:20.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.677+0000 7fcc70d32640 1 --2- 172.21.15.170:0/396104874 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc6c0b4d10 0x7fcc6c0aae60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcc680097a0 tx=0x7fcc6802b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b005e80667732720 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:20.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.677+0000 7fcc637fe640 1 -- 172.21.15.170:0/396104874 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc68031780 con 0x7fcc6c0b4d10 2024-05-08T04:13:20.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.677+0000 7fcc637fe640 1 -- 172.21.15.170:0/396104874 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc68037070 con 0x7fcc6c0b4d10 2024-05-08T04:13:20.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.677+0000 7fcc637fe640 1 -- 172.21.15.170:0/396104874 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc68031b40 con 0x7fcc6c0b4d10 2024-05-08T04:13:20.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.678+0000 7fcc72f96640 1 -- 172.21.15.170:0/396104874 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc6c0b4d10 msgr2=0x7fcc6c0aae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:20.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.678+0000 7fcc72f96640 1 --2- 172.21.15.170:0/396104874 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc6c0b4d10 0x7fcc6c0aae60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcc680097a0 tx=0x7fcc6802b820 comp rx=0 tx=0).stop 2024-05-08T04:13:20.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.678+0000 7fcc72f96640 1 -- 172.21.15.170:0/396104874 shutdown_connections 2024-05-08T04:13:20.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.678+0000 7fcc72f96640 1 --2- 172.21.15.170:0/396104874 >> v2:172.21.15.187:3300/0 conn(0x7fcc6c13bcc0 0x7fcc6c1459a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:20.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.678+0000 7fcc72f96640 1 --2- 172.21.15.170:0/396104874 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc6c0b4d10 0x7fcc6c0aae60 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:20.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.678+0000 7fcc72f96640 1 -- 172.21.15.170:0/396104874 >> 172.21.15.170:0/396104874 conn(0x7fcc6c0b7ec0 msgr2=0x7fcc6c0b4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:20.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.678+0000 7fcc72f96640 1 -- 172.21.15.170:0/396104874 shutdown_connections 2024-05-08T04:13:20.683 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.678+0000 7fcc72f96640 1 -- 172.21.15.170:0/396104874 wait complete. 2024-05-08T04:13:20.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.679+0000 7fcc72f96640 1 Processor -- start 2024-05-08T04:13:20.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.679+0000 7fcc72f96640 1 -- start start 2024-05-08T04:13:20.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.680+0000 7fcc72f96640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc6c0b4d10 0x7fcc6c1d6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:20.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.680+0000 7fcc72f96640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc6c13b860 0x7fcc6c13a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:20.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.680+0000 7fcc72f96640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc6c13bcc0 0x7fcc6c13c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:20.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.680+0000 7fcc72f96640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c142dc0 con 0x7fcc6c0b4d10 2024-05-08T04:13:20.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.680+0000 7fcc72f96640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c142c40 con 0x7fcc6c13bcc0 2024-05-08T04:13:20.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.680+0000 7fcc72f96640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fcc6c14c520 con 0x7fcc6c13b860 2024-05-08T04:13:20.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.680+0000 7fcc63fff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc6c13b860 0x7fcc6c13a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:20.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.680+0000 7fcc70d32640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc6c0b4d10 0x7fcc6c1d6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:20.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.680+0000 7fcc63fff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc6c13b860 0x7fcc6c13a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:48218/0 (socket says 172.21.15.170:48218) 2024-05-08T04:13:20.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.680+0000 7fcc63fff640 1 -- 172.21.15.170:0/2451020035 learned_addr learned my addr 172.21.15.170:0/2451020035 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:13:20.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.680+0000 7fcc71533640 1 --2- 172.21.15.170:0/2451020035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc6c13bcc0 0x7fcc6c13c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:20.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.680+0000 7fcc63fff640 1 -- 172.21.15.170:0/2451020035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc6c13bcc0 msgr2=0x7fcc6c13c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:20.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.680+0000 7fcc63fff640 1 --2- 172.21.15.170:0/2451020035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc6c13bcc0 0x7fcc6c13c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:20.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.680+0000 7fcc63fff640 1 -- 172.21.15.170:0/2451020035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc6c0b4d10 msgr2=0x7fcc6c1d6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:20.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.680+0000 7fcc63fff640 1 --2- 172.21.15.170:0/2451020035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc6c0b4d10 0x7fcc6c1d6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:20.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.680+0000 7fcc63fff640 1 -- 172.21.15.170:0/2451020035 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc6c13a740 con 0x7fcc6c13b860 2024-05-08T04:13:20.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.681+0000 7fcc70d32640 1 --2- 172.21.15.170:0/2451020035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc6c0b4d10 0x7fcc6c1d6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:13:20.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.680+0000 7fcc71533640 1 --2- 172.21.15.170:0/2451020035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc6c13bcc0 0x7fcc6c13c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:13:20.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.681+0000 7fcc63fff640 1 --2- 172.21.15.170:0/2451020035 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc6c13b860 0x7fcc6c13a200 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c00c4c0 tx=0x7fcc5c00c980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:20.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.681+0000 7fcc617fa640 1 -- 172.21.15.170:0/2451020035 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c018070 con 0x7fcc6c13b860 2024-05-08T04:13:20.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.681+0000 7fcc617fa640 1 -- 172.21.15.170:0/2451020035 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc5c007500 con 0x7fcc6c13b860 2024-05-08T04:13:20.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.681+0000 7fcc72f96640 1 -- 172.21.15.170:0/2451020035 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc6c13c6f0 con 0x7fcc6c13b860 2024-05-08T04:13:20.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.681+0000 7fcc617fa640 1 -- 172.21.15.170:0/2451020035 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c004a20 con 0x7fcc6c13b860 2024-05-08T04:13:20.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.681+0000 7fcc72f96640 1 -- 172.21.15.170:0/2451020035 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc6c1da7f0 con 0x7fcc6c13b860 2024-05-08T04:13:20.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.682+0000 7fcc72f96640 1 -- 172.21.15.170:0/2451020035 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc6c06e9e0 con 0x7fcc6c13b860 2024-05-08T04:13:20.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.683+0000 7fcc617fa640 1 -- 172.21.15.170:0/2451020035 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7fcc5c01b070 con 0x7fcc6c13b860 2024-05-08T04:13:20.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.683+0000 7fcc617fa640 1 --2- 172.21.15.170:0/2451020035 >> v2:172.21.15.187:6816/101007897 conn(0x7fcc580762c0 0x7fcc58078780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:20.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.683+0000 7fcc617fa640 1 -- 172.21.15.170:0/2451020035 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 4525+0+0 (secure 0 0 0) 0x7fcc5c0962a0 con 0x7fcc6c13b860 2024-05-08T04:13:20.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.683+0000 7fcc70d32640 1 --2- 172.21.15.170:0/2451020035 >> v2:172.21.15.187:6816/101007897 conn(0x7fcc580762c0 0x7fcc58078780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:20.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.684+0000 7fcc70d32640 1 --2- 172.21.15.170:0/2451020035 >> v2:172.21.15.187:6816/101007897 conn(0x7fcc580762c0 0x7fcc58078780 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcc68036000 tx=0x7fcc68008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:20.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.689+0000 7fcc617fa640 1 -- 172.21.15.170:0/2451020035 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fcc5c05fae0 con 0x7fcc6c13b860 2024-05-08T04:13:20.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.890+0000 7fcc72f96640 1 -- 172.21.15.170:0/2451020035 --> v2:172.21.15.187:6816/101007897 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fcc6c1da480 con 0x7fcc580762c0 2024-05-08T04:13:20.892 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.890+0000 7fcc617fa640 1 -- 172.21.15.170:0/2451020035 <== mgr.4107 v2:172.21.15.187:6816/101007897 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fcc6c1da480 con 0x7fcc580762c0 2024-05-08T04:13:20.892 INFO:teuthology.orchestra.run.smithi170.stderr:instructed osd(s) 7 to deep-scrub 2024-05-08T04:13:20.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.892+0000 7fcc72f96640 1 -- 172.21.15.170:0/2451020035 >> v2:172.21.15.187:6816/101007897 conn(0x7fcc580762c0 msgr2=0x7fcc58078780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:20.894 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.892+0000 7fcc72f96640 1 --2- 172.21.15.170:0/2451020035 >> v2:172.21.15.187:6816/101007897 conn(0x7fcc580762c0 0x7fcc58078780 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcc68036000 tx=0x7fcc68008030 comp rx=0 tx=0).stop 2024-05-08T04:13:20.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.892+0000 7fcc72f96640 1 -- 172.21.15.170:0/2451020035 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc6c13b860 msgr2=0x7fcc6c13a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:20.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.892+0000 7fcc72f96640 1 --2- 172.21.15.170:0/2451020035 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc6c13b860 0x7fcc6c13a200 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c00c4c0 tx=0x7fcc5c00c980 comp rx=0 tx=0).stop 2024-05-08T04:13:20.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.892+0000 7fcc72f96640 1 -- 172.21.15.170:0/2451020035 shutdown_connections 2024-05-08T04:13:20.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.892+0000 7fcc72f96640 1 --2- 172.21.15.170:0/2451020035 >> v2:172.21.15.187:6816/101007897 conn(0x7fcc580762c0 0x7fcc58078780 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:20.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.892+0000 7fcc72f96640 1 --2- 172.21.15.170:0/2451020035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc6c13bcc0 0x7fcc6c13c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:20.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.892+0000 7fcc72f96640 1 --2- 172.21.15.170:0/2451020035 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcc6c13b860 0x7fcc6c13a200 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:20.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.892+0000 7fcc72f96640 1 --2- 172.21.15.170:0/2451020035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc6c0b4d10 0x7fcc6c1d6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:20.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.892+0000 7fcc72f96640 1 -- 172.21.15.170:0/2451020035 >> 172.21.15.170:0/2451020035 conn(0x7fcc6c0b7ec0 msgr2=0x7fcc6c0b4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:20.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.892+0000 7fcc72f96640 1 -- 172.21.15.170:0/2451020035 shutdown_connections 2024-05-08T04:13:20.895 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:20.892+0000 7fcc72f96640 1 -- 172.21.15.170:0/2451020035 wait complete. 2024-05-08T04:13:20.907 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:13:21.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.008+0000 7f5cddb23640 1 Processor -- start 2024-05-08T04:13:21.010 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.008+0000 7f5cddb23640 1 -- start start 2024-05-08T04:13:21.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.008+0000 7f5cddb23640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5cd8142a20 0x7f5cd8071d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:21.013 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.008+0000 7f5cddb23640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cd80aafd0 con 0x7f5cd80aabf0 2024-05-08T04:13:21.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.008+0000 7f5cddb23640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cd80ac780 con 0x7f5cd80722b0 2024-05-08T04:13:21.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.008+0000 7f5cddb23640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5cd80ab8e0 con 0x7f5cd8142a20 2024-05-08T04:13:21.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.009+0000 7f5cd77fe640 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f5cd80aabf0 0x7f5cd8141ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43834/0 (socket says 172.21.15.170:43834) 2024-05-08T04:13:21.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.009+0000 7f5cd77fe640 1 -- 172.21.15.170:0/1817564632 learned_addr learned my addr 172.21.15.170:0/1817564632 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-05-08T04:13:21.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.009+0000 7f5ccffff640 1 --2- 172.21.15.170:0/1817564632 >> v2:172.21.15.187:3300/0 conn(0x7f5cd8142a20 0x7f5cd8071d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:21.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.009+0000 7f5cd6ffd640 1 -- 172.21.15.170:0/1817564632 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 91218069 0 0) 0x7f5cd80aafd0 con 0x7f5cd80aabf0 2024-05-08T04:13:21.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.009+0000 7f5cd6ffd640 1 -- 172.21.15.170:0/1817564632 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5cc4003520 con 0x7f5cd80aabf0 2024-05-08T04:13:21.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.009+0000 7f5ccffff640 1 -- 172.21.15.170:0/1817564632 >> v1:172.21.15.187:6789/0 conn(0x7f5cd80722b0 legacy=0x7f5cd8140c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:21.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.009+0000 7f5ccffff640 1 -- 172.21.15.170:0/1817564632 >> v1:172.21.15.170:6789/0 conn(0x7f5cd80aabf0 legacy=0x7f5cd8141ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:21.014 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.009+0000 7f5ccffff640 1 -- 172.21.15.170:0/1817564632 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd8142500 con 0x7f5cd8142a20 2024-05-08T04:13:21.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.010+0000 7f5ccffff640 1 --2- 172.21.15.170:0/1817564632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5cd8142a20 0x7f5cd8071d70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5cc00097a0 tx=0x7f5cc002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89f5692f75c60f49 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:21.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.010+0000 7f5cd6ffd640 1 -- 172.21.15.170:0/1817564632 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0035020 con 0x7f5cd8142a20 2024-05-08T04:13:21.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.010+0000 7f5cd6ffd640 1 -- 172.21.15.170:0/1817564632 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cc0031830 con 0x7f5cd8142a20 2024-05-08T04:13:21.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.010+0000 7f5cd6ffd640 1 -- 172.21.15.170:0/1817564632 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0031be0 con 0x7f5cd8142a20 2024-05-08T04:13:21.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.010+0000 7f5cddb23640 1 -- 172.21.15.170:0/1817564632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5cd8142a20 msgr2=0x7f5cd8071d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:21.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.010+0000 7f5cddb23640 1 --2- 172.21.15.170:0/1817564632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5cd8142a20 0x7f5cd8071d70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5cc00097a0 tx=0x7f5cc002b8d0 comp rx=0 tx=0).stop 2024-05-08T04:13:21.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.010+0000 7f5cddb23640 1 -- 172.21.15.170:0/1817564632 shutdown_connections 2024-05-08T04:13:21.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.010+0000 7f5cddb23640 1 --2- 172.21.15.170:0/1817564632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5cd8142a20 0x7f5cd8071d70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:21.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.010+0000 7f5cddb23640 1 -- 172.21.15.170:0/1817564632 >> 172.21.15.170:0/1817564632 conn(0x7f5cd80b7920 msgr2=0x7f5cd80b7d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:21.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.011+0000 7f5cddb23640 1 -- 172.21.15.170:0/1817564632 shutdown_connections 2024-05-08T04:13:21.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.011+0000 7f5cddb23640 1 -- 172.21.15.170:0/1817564632 wait complete. 2024-05-08T04:13:21.015 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.011+0000 7f5cddb23640 1 Processor -- start 2024-05-08T04:13:21.016 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.012+0000 7f5cddb23640 1 -- start start 2024-05-08T04:13:21.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.012+0000 7f5cddb23640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5cd80722b0 0x7f5cd8139c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:21.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.012+0000 7f5cddb23640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cd80aabf0 0x7f5cd81d7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:21.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.012+0000 7f5cddb23640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5cd8142a20 0x7f5cd80acf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:21.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.012+0000 7f5cddb23640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd81416c0 con 0x7f5cd80aabf0 2024-05-08T04:13:21.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.012+0000 7f5cddb23640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd8141540 con 0x7f5cd8142a20 2024-05-08T04:13:21.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.012+0000 7f5cddb23640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f5cd8141840 con 0x7f5cd80722b0 2024-05-08T04:13:21.019 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.013+0000 7f5cd77fe640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5cd80722b0 0x7f5cd8139c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:21.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.013+0000 7f5ccffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cd80aabf0 0x7f5cd81d7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:21.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.013+0000 7f5ccffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cd80aabf0 0x7f5cd81d7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49616/0 (socket says 172.21.15.170:49616) 2024-05-08T04:13:21.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.013+0000 7f5ccffff640 1 -- 172.21.15.170:0/1705493918 learned_addr learned my addr 172.21.15.170:0/1705493918 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:13:21.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.013+0000 7f5cd7fff640 1 --2- 172.21.15.170:0/1705493918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5cd8142a20 0x7f5cd80acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:21.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.013+0000 7f5ccffff640 1 -- 172.21.15.170:0/1705493918 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5cd80722b0 msgr2=0x7f5cd8139c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:21.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.013+0000 7f5ccffff640 1 --2- 172.21.15.170:0/1705493918 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5cd80722b0 0x7f5cd8139c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:21.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.013+0000 7f5ccffff640 1 -- 172.21.15.170:0/1705493918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5cd8142a20 msgr2=0x7f5cd80acf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:21.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.013+0000 7f5ccffff640 1 --2- 172.21.15.170:0/1705493918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5cd8142a20 0x7f5cd80acf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:21.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.013+0000 7f5ccffff640 1 -- 172.21.15.170:0/1705493918 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd80b0810 con 0x7f5cd80aabf0 2024-05-08T04:13:21.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.013+0000 7f5ccffff640 1 --2- 172.21.15.170:0/1705493918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cd80aabf0 0x7f5cd81d7760 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0034040 tx=0x7f5cc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:21.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.013+0000 7f5cd4ff9640 1 -- 172.21.15.170:0/1705493918 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0035040 con 0x7f5cd80aabf0 2024-05-08T04:13:21.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.013+0000 7f5cd4ff9640 1 -- 172.21.15.170:0/1705493918 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cc0004230 con 0x7f5cd80aabf0 2024-05-08T04:13:21.020 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.013+0000 7f5cddb23640 1 -- 172.21.15.170:0/1705493918 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cd80aec40 con 0x7f5cd80aabf0 2024-05-08T04:13:21.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.013+0000 7f5cd4ff9640 1 -- 172.21.15.170:0/1705493918 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc003a700 con 0x7f5cd80aabf0 2024-05-08T04:13:21.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.013+0000 7f5cddb23640 1 -- 172.21.15.170:0/1705493918 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cd81d6c90 con 0x7f5cd80aabf0 2024-05-08T04:13:21.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.014+0000 7f5cddb23640 1 -- 172.21.15.170:0/1705493918 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cd806e9e0 con 0x7f5cd80aabf0 2024-05-08T04:13:21.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.015+0000 7f5cd4ff9640 1 -- 172.21.15.170:0/1705493918 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f5cc0038020 con 0x7f5cd80aabf0 2024-05-08T04:13:21.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.016+0000 7f5cd4ff9640 1 --2- 172.21.15.170:0/1705493918 >> v2:172.21.15.187:6816/101007897 conn(0x7f5cb8076290 0x7f5cb8078750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:21.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.016+0000 7f5cd4ff9640 1 -- 172.21.15.170:0/1705493918 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 4525+0+0 (secure 0 0 0) 0x7f5cc00b8d60 con 0x7f5cd80aabf0 2024-05-08T04:13:21.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.016+0000 7f5cd77fe640 1 --2- 172.21.15.170:0/1705493918 >> v2:172.21.15.187:6816/101007897 conn(0x7f5cb8076290 0x7f5cb8078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:21.021 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.016+0000 7f5cd77fe640 1 --2- 172.21.15.170:0/1705493918 >> v2:172.21.15.187:6816/101007897 conn(0x7f5cb8076290 0x7f5cb8078750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8005880 tx=0x7f5cc8007cd0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:21.022 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.020+0000 7f5cd4ff9640 1 -- 172.21.15.170:0/1705493918 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f5cc0082520 con 0x7f5cd80aabf0 2024-05-08T04:13:21.224 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.222+0000 7f5cddb23640 1 -- 172.21.15.170:0/1705493918 --> v2:172.21.15.187:6816/101007897 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5cd806b400 con 0x7f5cb8076290 2024-05-08T04:13:21.227 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.226+0000 7f5cd4ff9640 1 -- 172.21.15.170:0/1705493918 <== mgr.4107 v2:172.21.15.187:6816/101007897 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+117781 (secure 0 0 0) 0x7f5cd806b400 con 0x7f5cb8076290 2024-05-08T04:13:21.228 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:13:21.229 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-05-08T04:13:21.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.228+0000 7f5cddb23640 1 -- 172.21.15.170:0/1705493918 >> v2:172.21.15.187:6816/101007897 conn(0x7f5cb8076290 msgr2=0x7f5cb8078750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:21.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.228+0000 7f5cddb23640 1 --2- 172.21.15.170:0/1705493918 >> v2:172.21.15.187:6816/101007897 conn(0x7f5cb8076290 0x7f5cb8078750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8005880 tx=0x7f5cc8007cd0 comp rx=0 tx=0).stop 2024-05-08T04:13:21.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.228+0000 7f5cddb23640 1 -- 172.21.15.170:0/1705493918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cd80aabf0 msgr2=0x7f5cd81d7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:21.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.228+0000 7f5cddb23640 1 --2- 172.21.15.170:0/1705493918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cd80aabf0 0x7f5cd81d7760 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0034040 tx=0x7f5cc00046c0 comp rx=0 tx=0).stop 2024-05-08T04:13:21.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.229+0000 7f5cddb23640 1 -- 172.21.15.170:0/1705493918 shutdown_connections 2024-05-08T04:13:21.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.229+0000 7f5cddb23640 1 --2- 172.21.15.170:0/1705493918 >> v2:172.21.15.187:6816/101007897 conn(0x7f5cb8076290 0x7f5cb8078750 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:21.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.229+0000 7f5cddb23640 1 --2- 172.21.15.170:0/1705493918 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5cd8142a20 0x7f5cd80acf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:21.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.229+0000 7f5cddb23640 1 --2- 172.21.15.170:0/1705493918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cd80aabf0 0x7f5cd81d7760 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:21.231 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.229+0000 7f5cddb23640 1 --2- 172.21.15.170:0/1705493918 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f5cd80722b0 0x7f5cd8139c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:21.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.229+0000 7f5cddb23640 1 -- 172.21.15.170:0/1705493918 >> 172.21.15.170:0/1705493918 conn(0x7f5cd80b7920 msgr2=0x7f5cd80732c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:21.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.229+0000 7f5cddb23640 1 -- 172.21.15.170:0/1705493918 shutdown_connections 2024-05-08T04:13:21.232 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:21.229+0000 7f5cddb23640 1 -- 172.21.15.170:0/1705493918 wait complete. 2024-05-08T04:13:21.245 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":167,"stamp":"2024-05-08T04:13:19.720309+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3581797144,"num_objects":882,"num_object_clones":0,"num_object_copies":1764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":882,"num_whiteouts":0,"num_read":2636,"num_read_kb":2641,"num_write":164626,"num_write_kb":5657401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167333,"ondisk_log_size":167333,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9338204,"kb_used_data":6665348,"kb_used_omap":12,"kb_used_meta":2672755,"kb_avail":745636516,"statfs":{"total":773094113280,"available":763531792384,"internally_reserved":0,"allocated":6825316352,"data_stored":6862705696,"data_compressed":194295728,"data_compressed_allocated":202113024,"data_compressed_original":244916224,"omap_allocated":12751,"internal_metadata":2736901681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8720,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":66,"num_read_kb":71,"num_write":17,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002641"},"pg_stats":[{"pgid":"3.1f","version":"21'3951","reported_seq":3986,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.244099+0000","last_change":"2024-05-08T04:08:27.732101+0000","last_active":"2024-05-08T04:13:14.244099+0000","last_peered":"2024-05-08T04:13:14.244099+0000","last_clean":"2024-05-08T04:13:14.244099+0000","last_became_active":"2024-05-08T04:08:27.731781+0000","last_became_peered":"2024-05-08T04:08:27.731781+0000","last_unstale":"2024-05-08T04:13:14.244099+0000","last_undegraded":"2024-05-08T04:13:14.244099+0000","last_fullsized":"2024-05-08T04:13:14.244099+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":3951,"log_dups_size":0,"ondisk_log_size":3951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:46:39.944802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":62,"num_read_kb":52,"num_write":3964,"num_write_kb":169350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"23'7381","reported_seq":7401,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.243557+0000","last_change":"2024-05-08T04:08:27.731984+0000","last_active":"2024-05-08T04:13:14.243557+0000","last_peered":"2024-05-08T04:13:14.243557+0000","last_clean":"2024-05-08T04:13:14.243557+0000","last_became_active":"2024-05-08T04:08:27.731598+0000","last_became_peered":"2024-05-08T04:08:27.731598+0000","last_unstale":"2024-05-08T04:13:14.243557+0000","last_undegraded":"2024-05-08T04:13:14.243557+0000","last_fullsized":"2024-05-08T04:13:14.243557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":7381,"log_dups_size":0,"ondisk_log_size":7381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:41:22.212562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":91594752,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":7365,"num_write_kb":213614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"23'5765","reported_seq":5796,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.282559+0000","last_change":"2024-05-08T04:08:27.732284+0000","last_active":"2024-05-08T04:13:14.282559+0000","last_peered":"2024-05-08T04:13:14.282559+0000","last_clean":"2024-05-08T04:13:14.282559+0000","last_became_active":"2024-05-08T04:08:27.732006+0000","last_became_peered":"2024-05-08T04:08:27.732006+0000","last_unstale":"2024-05-08T04:13:14.282559+0000","last_undegraded":"2024-05-08T04:13:14.282559+0000","last_fullsized":"2024-05-08T04:13:14.282559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":5765,"log_dups_size":0,"ondisk_log_size":5765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:24:55.281049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":56,"num_read_kb":44,"num_write":5770,"num_write_kb":177284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"21'4466","reported_seq":4490,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.280830+0000","last_change":"2024-05-08T04:08:27.733083+0000","last_active":"2024-05-08T04:13:14.280830+0000","last_peered":"2024-05-08T04:13:14.280830+0000","last_clean":"2024-05-08T04:13:14.280830+0000","last_became_active":"2024-05-08T04:08:27.732710+0000","last_became_peered":"2024-05-08T04:08:27.732710+0000","last_unstale":"2024-05-08T04:13:14.280830+0000","last_undegraded":"2024-05-08T04:13:14.280830+0000","last_fullsized":"2024-05-08T04:13:14.280830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":4466,"log_dups_size":0,"ondisk_log_size":4466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:40:43.219199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125829120,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":4481,"num_write_kb":186066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"23'4175","reported_seq":4194,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.283656+0000","last_change":"2024-05-08T04:08:27.726522+0000","last_active":"2024-05-08T04:13:14.283656+0000","last_peered":"2024-05-08T04:13:14.283656+0000","last_clean":"2024-05-08T04:13:14.283656+0000","last_became_active":"2024-05-08T04:08:27.726232+0000","last_became_peered":"2024-05-08T04:08:27.726232+0000","last_unstale":"2024-05-08T04:13:14.283656+0000","last_undegraded":"2024-05-08T04:13:14.283656+0000","last_fullsized":"2024-05-08T04:13:14.283656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":4175,"log_dups_size":0,"ondisk_log_size":4175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:07.708671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120203616,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":4136,"num_write_kb":200298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"23'6141","reported_seq":6161,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.246780+0000","last_change":"2024-05-08T04:08:27.722514+0000","last_active":"2024-05-08T04:13:14.246780+0000","last_peered":"2024-05-08T04:13:14.246780+0000","last_clean":"2024-05-08T04:13:14.246780+0000","last_became_active":"2024-05-08T04:08:27.721701+0000","last_became_peered":"2024-05-08T04:08:27.721701+0000","last_unstale":"2024-05-08T04:13:14.246780+0000","last_undegraded":"2024-05-08T04:13:14.246780+0000","last_fullsized":"2024-05-08T04:13:14.246780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":6141,"log_dups_size":0,"ondisk_log_size":6141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:44.554641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":6123,"num_write_kb":209934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"23'5187","reported_seq":5207,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.286850+0000","last_change":"2024-05-08T04:08:27.728824+0000","last_active":"2024-05-08T04:13:14.286850+0000","last_peered":"2024-05-08T04:13:14.286850+0000","last_clean":"2024-05-08T04:13:14.286850+0000","last_became_active":"2024-05-08T04:08:27.728564+0000","last_became_peered":"2024-05-08T04:08:27.728564+0000","last_unstale":"2024-05-08T04:13:14.286850+0000","last_undegraded":"2024-05-08T04:13:14.286850+0000","last_fullsized":"2024-05-08T04:13:14.286850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":5187,"log_dups_size":0,"ondisk_log_size":5187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:55.969304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142606336,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":5146,"num_write_kb":193669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"22'7560","reported_seq":7581,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.243068+0000","last_change":"2024-05-08T04:08:27.728565+0000","last_active":"2024-05-08T04:13:14.243068+0000","last_peered":"2024-05-08T04:13:14.243068+0000","last_clean":"2024-05-08T04:13:14.243068+0000","last_became_active":"2024-05-08T04:08:27.728205+0000","last_became_peered":"2024-05-08T04:08:27.728205+0000","last_unstale":"2024-05-08T04:13:14.243068+0000","last_undegraded":"2024-05-08T04:13:14.243068+0000","last_fullsized":"2024-05-08T04:13:14.243068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":7560,"log_dups_size":0,"ondisk_log_size":7560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:22:53.989163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142606336,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":12,"num_read_kb":135,"num_write":7569,"num_write_kb":191194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"22'5903","reported_seq":5920,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.244254+0000","last_change":"2024-05-08T04:08:27.732104+0000","last_active":"2024-05-08T04:13:14.244254+0000","last_peered":"2024-05-08T04:13:14.244254+0000","last_clean":"2024-05-08T04:13:14.244254+0000","last_became_active":"2024-05-08T04:08:27.731769+0000","last_became_peered":"2024-05-08T04:08:27.731769+0000","last_unstale":"2024-05-08T04:13:14.244254+0000","last_undegraded":"2024-05-08T04:13:14.244254+0000","last_fullsized":"2024-05-08T04:13:14.244254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":5903,"log_dups_size":0,"ondisk_log_size":5903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:21:26.194969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5896,"num_write_kb":160193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"23'4608","reported_seq":4626,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.283448+0000","last_change":"2024-05-08T04:08:27.731045+0000","last_active":"2024-05-08T04:13:14.283448+0000","last_peered":"2024-05-08T04:13:14.283448+0000","last_clean":"2024-05-08T04:13:14.283448+0000","last_became_active":"2024-05-08T04:08:27.730749+0000","last_became_peered":"2024-05-08T04:08:27.730749+0000","last_unstale":"2024-05-08T04:13:14.283448+0000","last_undegraded":"2024-05-08T04:13:14.283448+0000","last_fullsized":"2024-05-08T04:13:14.283448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":4608,"log_dups_size":0,"ondisk_log_size":4608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:17:07.247072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":127271424,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":4592,"num_write_kb":174343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"22'7467","reported_seq":7795,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.243724+0000","last_change":"2024-05-08T04:08:27.729609+0000","last_active":"2024-05-08T04:13:14.243724+0000","last_peered":"2024-05-08T04:13:14.243724+0000","last_clean":"2024-05-08T04:13:14.243724+0000","last_became_active":"2024-05-08T04:08:27.729318+0000","last_became_peered":"2024-05-08T04:08:27.729318+0000","last_unstale":"2024-05-08T04:13:14.243724+0000","last_undegraded":"2024-05-08T04:13:14.243724+0000","last_fullsized":"2024-05-08T04:13:14.243724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":7467,"log_dups_size":0,"ondisk_log_size":7467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:17:45.553221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112068432,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1045,"num_read_kb":996,"num_write":7685,"num_write_kb":175598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"23'6087","reported_seq":6105,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.290448+0000","last_change":"2024-05-08T04:08:27.730319+0000","last_active":"2024-05-08T04:13:14.290448+0000","last_peered":"2024-05-08T04:13:14.290448+0000","last_clean":"2024-05-08T04:13:14.290448+0000","last_became_active":"2024-05-08T04:08:27.729781+0000","last_became_peered":"2024-05-08T04:08:27.729781+0000","last_unstale":"2024-05-08T04:13:14.290448+0000","last_undegraded":"2024-05-08T04:13:14.290448+0000","last_fullsized":"2024-05-08T04:13:14.290448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":6087,"log_dups_size":0,"ondisk_log_size":6087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:29:26.256991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123143848,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6035,"num_write_kb":178933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.280716+0000","last_change":"2024-05-08T04:08:21.697423+0000","last_active":"2024-05-08T04:13:14.280716+0000","last_peered":"2024-05-08T04:13:14.280716+0000","last_clean":"2024-05-08T04:13:14.280716+0000","last_became_active":"2024-05-08T04:08:21.697149+0000","last_became_peered":"2024-05-08T04:08:21.697149+0000","last_unstale":"2024-05-08T04:13:14.280716+0000","last_undegraded":"2024-05-08T04:13:14.280716+0000","last_fullsized":"2024-05-08T04:13:14.280716+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:59:38.354372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'7340","reported_seq":7381,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.244087+0000","last_change":"2024-05-08T04:08:27.725473+0000","last_active":"2024-05-08T04:13:14.244087+0000","last_peered":"2024-05-08T04:13:14.244087+0000","last_clean":"2024-05-08T04:13:14.244087+0000","last_became_active":"2024-05-08T04:08:27.725165+0000","last_became_peered":"2024-05-08T04:08:27.725165+0000","last_unstale":"2024-05-08T04:13:14.244087+0000","last_undegraded":"2024-05-08T04:13:14.244087+0000","last_fullsized":"2024-05-08T04:13:14.244087+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":7340,"log_dups_size":0,"ondisk_log_size":7340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:06:19.688474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142738448,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":74,"num_read_kb":59,"num_write":7371,"num_write_kb":187653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.278833+0000","last_change":"2024-05-08T04:08:21.696942+0000","last_active":"2024-05-08T04:13:14.278833+0000","last_peered":"2024-05-08T04:13:14.278833+0000","last_clean":"2024-05-08T04:13:14.278833+0000","last_became_active":"2024-05-08T04:08:21.696632+0000","last_became_peered":"2024-05-08T04:08:21.696632+0000","last_unstale":"2024-05-08T04:13:14.278833+0000","last_undegraded":"2024-05-08T04:13:14.278833+0000","last_fullsized":"2024-05-08T04:13:14.278833+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:23:25.156383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'3712","reported_seq":3730,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.289760+0000","last_change":"2024-05-08T04:08:27.730834+0000","last_active":"2024-05-08T04:13:14.289760+0000","last_peered":"2024-05-08T04:13:14.289760+0000","last_clean":"2024-05-08T04:13:14.289760+0000","last_became_active":"2024-05-08T04:08:27.730530+0000","last_became_peered":"2024-05-08T04:08:27.730530+0000","last_unstale":"2024-05-08T04:13:14.289760+0000","last_undegraded":"2024-05-08T04:13:14.289760+0000","last_fullsized":"2024-05-08T04:13:14.289760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":3712,"log_dups_size":0,"ondisk_log_size":3712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:51:02.242727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3661,"num_write_kb":168963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.244054+0000","last_change":"2024-05-08T04:08:21.696814+0000","last_active":"2024-05-08T04:13:14.244054+0000","last_peered":"2024-05-08T04:13:14.244054+0000","last_clean":"2024-05-08T04:13:14.244054+0000","last_became_active":"2024-05-08T04:08:21.696403+0000","last_became_peered":"2024-05-08T04:08:21.696403+0000","last_unstale":"2024-05-08T04:13:14.244054+0000","last_undegraded":"2024-05-08T04:13:14.244054+0000","last_fullsized":"2024-05-08T04:13:14.244054+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:02:04.075137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"22'6934","reported_seq":6952,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.278367+0000","last_change":"2024-05-08T04:08:27.721621+0000","last_active":"2024-05-08T04:13:14.278367+0000","last_peered":"2024-05-08T04:13:14.278367+0000","last_clean":"2024-05-08T04:13:14.278367+0000","last_became_active":"2024-05-08T04:08:27.721303+0000","last_became_peered":"2024-05-08T04:08:27.721303+0000","last_unstale":"2024-05-08T04:13:14.278367+0000","last_undegraded":"2024-05-08T04:13:14.278367+0000","last_fullsized":"2024-05-08T04:13:14.278367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":6934,"log_dups_size":0,"ondisk_log_size":6934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:59:37.780694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6930,"num_write_kb":193544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.278665+0000","last_change":"2024-05-08T04:08:21.693307+0000","last_active":"2024-05-08T04:13:14.278665+0000","last_peered":"2024-05-08T04:13:14.278665+0000","last_clean":"2024-05-08T04:13:14.278665+0000","last_became_active":"2024-05-08T04:08:21.693014+0000","last_became_peered":"2024-05-08T04:08:21.693014+0000","last_unstale":"2024-05-08T04:13:14.278665+0000","last_undegraded":"2024-05-08T04:13:14.278665+0000","last_fullsized":"2024-05-08T04:13:14.278665+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:56.218680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"22'4514","reported_seq":4533,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.282828+0000","last_change":"2024-05-08T04:08:27.729798+0000","last_active":"2024-05-08T04:13:14.282828+0000","last_peered":"2024-05-08T04:13:14.282828+0000","last_clean":"2024-05-08T04:13:14.282828+0000","last_became_active":"2024-05-08T04:08:27.729504+0000","last_became_peered":"2024-05-08T04:08:27.729504+0000","last_unstale":"2024-05-08T04:13:14.282828+0000","last_undegraded":"2024-05-08T04:13:14.282828+0000","last_fullsized":"2024-05-08T04:13:14.282828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":4514,"log_dups_size":0,"ondisk_log_size":4514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:58:47.867425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96535048,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":4515,"num_write_kb":160844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"21'6272","reported_seq":6403,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.246844+0000","last_change":"2024-05-08T04:08:27.726831+0000","last_active":"2024-05-08T04:13:14.246844+0000","last_peered":"2024-05-08T04:13:14.246844+0000","last_clean":"2024-05-08T04:13:14.246844+0000","last_became_active":"2024-05-08T04:08:27.726527+0000","last_became_peered":"2024-05-08T04:08:27.726527+0000","last_unstale":"2024-05-08T04:13:14.246844+0000","last_undegraded":"2024-05-08T04:13:14.246844+0000","last_fullsized":"2024-05-08T04:13:14.246844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":6272,"log_dups_size":0,"ondisk_log_size":6272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:26:39.243499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":126223376,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":153,"num_read_kb":139,"num_write":6308,"num_write_kb":247140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.280609+0000","last_change":"2024-05-08T04:08:21.699221+0000","last_active":"2024-05-08T04:13:14.280609+0000","last_peered":"2024-05-08T04:13:14.280609+0000","last_clean":"2024-05-08T04:13:14.280609+0000","last_became_active":"2024-05-08T04:08:21.699013+0000","last_became_peered":"2024-05-08T04:08:21.699013+0000","last_unstale":"2024-05-08T04:13:14.280609+0000","last_undegraded":"2024-05-08T04:13:14.280609+0000","last_fullsized":"2024-05-08T04:13:14.280609+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:23:08.040183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"23'3069","reported_seq":3087,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.285582+0000","last_change":"2024-05-08T04:08:27.723409+0000","last_active":"2024-05-08T04:13:14.285582+0000","last_peered":"2024-05-08T04:13:14.285582+0000","last_clean":"2024-05-08T04:13:14.285582+0000","last_became_active":"2024-05-08T04:08:27.723118+0000","last_became_peered":"2024-05-08T04:08:27.723118+0000","last_unstale":"2024-05-08T04:13:14.285582+0000","last_undegraded":"2024-05-08T04:13:14.285582+0000","last_fullsized":"2024-05-08T04:13:14.285582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":3069,"log_dups_size":0,"ondisk_log_size":3069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:31:49.593896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3019,"num_write_kb":152195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.251393+0000","last_change":"2024-05-08T04:08:21.799283+0000","last_active":"2024-05-08T04:13:14.251393+0000","last_peered":"2024-05-08T04:13:14.251393+0000","last_clean":"2024-05-08T04:13:14.251393+0000","last_became_active":"2024-05-08T04:08:21.798995+0000","last_became_peered":"2024-05-08T04:08:21.798995+0000","last_unstale":"2024-05-08T04:13:14.251393+0000","last_undegraded":"2024-05-08T04:13:14.251393+0000","last_fullsized":"2024-05-08T04:13:14.251393+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:37:16.795888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"23'3229","reported_seq":3248,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.243180+0000","last_change":"2024-05-08T04:08:27.726742+0000","last_active":"2024-05-08T04:13:14.243180+0000","last_peered":"2024-05-08T04:13:14.243180+0000","last_clean":"2024-05-08T04:13:14.243180+0000","last_became_active":"2024-05-08T04:08:27.726457+0000","last_became_peered":"2024-05-08T04:08:27.726457+0000","last_unstale":"2024-05-08T04:13:14.243180+0000","last_undegraded":"2024-05-08T04:13:14.243180+0000","last_fullsized":"2024-05-08T04:13:14.243180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":3229,"log_dups_size":0,"ondisk_log_size":3229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:36:59.353164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104923656,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":3211,"num_write_kb":154644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.243505+0000","last_change":"2024-05-08T04:08:21.697070+0000","last_active":"2024-05-08T04:13:14.243505+0000","last_peered":"2024-05-08T04:13:14.243505+0000","last_clean":"2024-05-08T04:13:14.243505+0000","last_became_active":"2024-05-08T04:08:21.696370+0000","last_became_peered":"2024-05-08T04:08:21.696370+0000","last_unstale":"2024-05-08T04:13:14.243505+0000","last_undegraded":"2024-05-08T04:13:14.243505+0000","last_fullsized":"2024-05-08T04:13:14.243505+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:23:54.981822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"22'5074","reported_seq":5092,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.243183+0000","last_change":"2024-05-08T04:08:27.732812+0000","last_active":"2024-05-08T04:13:14.243183+0000","last_peered":"2024-05-08T04:13:14.243183+0000","last_clean":"2024-05-08T04:13:14.243183+0000","last_became_active":"2024-05-08T04:08:27.732446+0000","last_became_peered":"2024-05-08T04:08:27.732446+0000","last_unstale":"2024-05-08T04:13:14.243183+0000","last_undegraded":"2024-05-08T04:13:14.243183+0000","last_fullsized":"2024-05-08T04:13:14.243183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":5074,"log_dups_size":0,"ondisk_log_size":5074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:34:38.276980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5070,"num_write_kb":160386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.280726+0000","last_change":"2024-05-08T04:08:21.696942+0000","last_active":"2024-05-08T04:13:14.280726+0000","last_peered":"2024-05-08T04:13:14.280726+0000","last_clean":"2024-05-08T04:13:14.280726+0000","last_became_active":"2024-05-08T04:08:21.696659+0000","last_became_peered":"2024-05-08T04:08:21.696659+0000","last_unstale":"2024-05-08T04:13:14.280726+0000","last_undegraded":"2024-05-08T04:13:14.280726+0000","last_fullsized":"2024-05-08T04:13:14.280726+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:20.683987+0000","last_clean_scrub_stamp":"2024-05-08T04:08:20.683987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:50:44.209520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'72","reported_seq":116,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.243514+0000","last_change":"2024-05-08T04:08:20.691470+0000","last_active":"2024-05-08T04:13:14.243514+0000","last_peered":"2024-05-08T04:13:14.243514+0000","last_clean":"2024-05-08T04:13:14.243514+0000","last_became_active":"2024-05-08T04:08:20.691204+0000","last_became_peered":"2024-05-08T04:08:20.691204+0000","last_unstale":"2024-05-08T04:13:14.243514+0000","last_undegraded":"2024-05-08T04:13:14.243514+0000","last_fullsized":"2024-05-08T04:13:14.243514+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:19.679313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:19.679313+0000","last_clean_scrub_stamp":"2024-05-08T04:08:19.679313+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:52:31.185659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"23'3940","reported_seq":3963,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.285959+0000","last_change":"2024-05-08T04:08:27.729757+0000","last_active":"2024-05-08T04:13:14.285959+0000","last_peered":"2024-05-08T04:13:14.285959+0000","last_clean":"2024-05-08T04:13:14.285959+0000","last_became_active":"2024-05-08T04:08:27.729321+0000","last_became_peered":"2024-05-08T04:08:27.729321+0000","last_unstale":"2024-05-08T04:13:14.285959+0000","last_undegraded":"2024-05-08T04:13:14.285959+0000","last_fullsized":"2024-05-08T04:13:14.285959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":3940,"log_dups_size":0,"ondisk_log_size":3940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:12:28.413005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109182976,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":3879,"num_write_kb":165315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"23'9391","reported_seq":9539,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.311352+0000","last_change":"2024-05-08T04:08:27.721874+0000","last_active":"2024-05-08T04:13:14.311352+0000","last_peered":"2024-05-08T04:13:14.311352+0000","last_clean":"2024-05-08T04:13:14.311352+0000","last_became_active":"2024-05-08T04:08:27.721534+0000","last_became_peered":"2024-05-08T04:08:27.721534+0000","last_unstale":"2024-05-08T04:13:14.311352+0000","last_undegraded":"2024-05-08T04:13:14.311352+0000","last_fullsized":"2024-05-08T04:13:14.311352+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":9391,"log_dups_size":0,"ondisk_log_size":9391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:59:46.563205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":139724624,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":407,"num_read_kb":368,"num_write":6462,"num_write_kb":194675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"23'3828","reported_seq":3868,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.280742+0000","last_change":"2024-05-08T04:08:27.726164+0000","last_active":"2024-05-08T04:13:14.280742+0000","last_peered":"2024-05-08T04:13:14.280742+0000","last_clean":"2024-05-08T04:13:14.280742+0000","last_became_active":"2024-05-08T04:08:27.725833+0000","last_became_peered":"2024-05-08T04:08:27.725833+0000","last_unstale":"2024-05-08T04:13:14.280742+0000","last_undegraded":"2024-05-08T04:13:14.280742+0000","last_fullsized":"2024-05-08T04:13:14.280742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":3828,"log_dups_size":0,"ondisk_log_size":3828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:35.399549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":85458944,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":91,"num_read_kb":137,"num_write":3837,"num_write_kb":155137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"21'4041","reported_seq":4083,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.243142+0000","last_change":"2024-05-08T04:08:27.727947+0000","last_active":"2024-05-08T04:13:14.243142+0000","last_peered":"2024-05-08T04:13:14.243142+0000","last_clean":"2024-05-08T04:13:14.243142+0000","last_became_active":"2024-05-08T04:08:27.727653+0000","last_became_peered":"2024-05-08T04:08:27.727653+0000","last_unstale":"2024-05-08T04:13:14.243142+0000","last_undegraded":"2024-05-08T04:13:14.243142+0000","last_fullsized":"2024-05-08T04:13:14.243142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":4041,"log_dups_size":0,"ondisk_log_size":4041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:14:37.705194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130023424,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":90,"num_read_kb":141,"num_write":4071,"num_write_kb":155757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"23'5324","reported_seq":5350,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.291432+0000","last_change":"2024-05-08T04:08:27.730956+0000","last_active":"2024-05-08T04:13:14.291432+0000","last_peered":"2024-05-08T04:13:14.291432+0000","last_clean":"2024-05-08T04:13:14.291432+0000","last_became_active":"2024-05-08T04:08:27.730569+0000","last_became_peered":"2024-05-08T04:08:27.730569+0000","last_unstale":"2024-05-08T04:13:14.291432+0000","last_undegraded":"2024-05-08T04:13:14.291432+0000","last_fullsized":"2024-05-08T04:13:14.291432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":5324,"log_dups_size":0,"ondisk_log_size":5324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:05:40.969984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":5289,"num_write_kb":148492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"23'4024","reported_seq":4056,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.243696+0000","last_change":"2024-05-08T04:08:27.733357+0000","last_active":"2024-05-08T04:13:14.243696+0000","last_peered":"2024-05-08T04:13:14.243696+0000","last_clean":"2024-05-08T04:13:14.243696+0000","last_became_active":"2024-05-08T04:08:27.733070+0000","last_became_peered":"2024-05-08T04:08:27.733070+0000","last_unstale":"2024-05-08T04:13:14.243696+0000","last_undegraded":"2024-05-08T04:13:14.243696+0000","last_fullsized":"2024-05-08T04:13:14.243696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":4024,"log_dups_size":0,"ondisk_log_size":4024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:33:08.617696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":103481344,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":53,"num_read_kb":105,"num_write":4128,"num_write_kb":186691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"23'3223","reported_seq":3343,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.298622+0000","last_change":"2024-05-08T04:08:27.730671+0000","last_active":"2024-05-08T04:13:14.298622+0000","last_peered":"2024-05-08T04:13:14.298622+0000","last_clean":"2024-05-08T04:13:14.298622+0000","last_became_active":"2024-05-08T04:08:27.730372+0000","last_became_peered":"2024-05-08T04:08:27.730372+0000","last_unstale":"2024-05-08T04:13:14.298622+0000","last_undegraded":"2024-05-08T04:13:14.298622+0000","last_fullsized":"2024-05-08T04:13:14.298622+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":3223,"log_dups_size":0,"ondisk_log_size":3223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:55:59.085956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":78381056,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":342,"num_read_kb":267,"num_write":3496,"num_write_kb":113772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"23'4349","reported_seq":4370,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.289521+0000","last_change":"2024-05-08T04:08:27.729948+0000","last_active":"2024-05-08T04:13:14.289521+0000","last_peered":"2024-05-08T04:13:14.289521+0000","last_clean":"2024-05-08T04:13:14.289521+0000","last_became_active":"2024-05-08T04:08:27.729584+0000","last_became_peered":"2024-05-08T04:08:27.729584+0000","last_unstale":"2024-05-08T04:13:14.289521+0000","last_undegraded":"2024-05-08T04:13:14.289521+0000","last_fullsized":"2024-05-08T04:13:14.289521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":4349,"log_dups_size":0,"ondisk_log_size":4349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:35:18.017771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95482384,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":4309,"num_write_kb":147779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"23'5306","reported_seq":5332,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.291888+0000","last_change":"2024-05-08T04:08:27.727021+0000","last_active":"2024-05-08T04:13:14.291888+0000","last_peered":"2024-05-08T04:13:14.291888+0000","last_clean":"2024-05-08T04:13:14.291888+0000","last_became_active":"2024-05-08T04:08:27.726692+0000","last_became_peered":"2024-05-08T04:08:27.726692+0000","last_unstale":"2024-05-08T04:13:14.291888+0000","last_undegraded":"2024-05-08T04:13:14.291888+0000","last_fullsized":"2024-05-08T04:13:14.291888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":5306,"log_dups_size":0,"ondisk_log_size":5306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:16:39.270048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120979456,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":5269,"num_write_kb":245378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"21'2450","reported_seq":2468,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.243820+0000","last_change":"2024-05-08T04:08:27.727459+0000","last_active":"2024-05-08T04:13:14.243820+0000","last_peered":"2024-05-08T04:13:14.243820+0000","last_clean":"2024-05-08T04:13:14.243820+0000","last_became_active":"2024-05-08T04:08:27.727138+0000","last_became_peered":"2024-05-08T04:08:27.727138+0000","last_unstale":"2024-05-08T04:13:14.243820+0000","last_undegraded":"2024-05-08T04:13:14.243820+0000","last_fullsized":"2024-05-08T04:13:14.243820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":2450,"log_dups_size":0,"ondisk_log_size":2450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:32:43.624016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2442,"num_write_kb":115332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"23'3790","reported_seq":3816,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.289583+0000","last_change":"2024-05-08T04:08:27.729355+0000","last_active":"2024-05-08T04:13:14.289583+0000","last_peered":"2024-05-08T04:13:14.289583+0000","last_clean":"2024-05-08T04:13:14.289583+0000","last_became_active":"2024-05-08T04:08:27.729024+0000","last_became_peered":"2024-05-08T04:08:27.729024+0000","last_unstale":"2024-05-08T04:13:14.289583+0000","last_undegraded":"2024-05-08T04:13:14.289583+0000","last_fullsized":"2024-05-08T04:13:14.289583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":3790,"log_dups_size":0,"ondisk_log_size":3790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:28:16.629952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96600064,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":3752,"num_write_kb":156932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"23'8760","reported_seq":8780,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:14.243612+0000","last_change":"2024-05-08T04:08:27.729279+0000","last_active":"2024-05-08T04:13:14.243612+0000","last_peered":"2024-05-08T04:13:14.243612+0000","last_clean":"2024-05-08T04:13:14.243612+0000","last_became_active":"2024-05-08T04:08:27.728925+0000","last_became_peered":"2024-05-08T04:08:27.728925+0000","last_unstale":"2024-05-08T04:13:14.243612+0000","last_undegraded":"2024-05-08T04:13:14.243612+0000","last_fullsized":"2024-05-08T04:13:14.243612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:08:26.712802+0000","last_clean_scrub_stamp":"2024-05-08T04:08:26.712802+0000","objects_scrubbed":0,"log_size":8760,"log_dups_size":0,"ondisk_log_size":8760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:40:38.725967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163709456,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":8744,"num_write_kb":214668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3581206776,"num_objects":880,"num_object_clones":0,"num_object_copies":1760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":880,"num_whiteouts":0,"num_read":2570,"num_read_kb":2586,"num_write":164525,"num_write_kb":5655773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6823485440,"data_stored":6861063288,"data_compressed":194285506,"data_compressed_allocated":202047488,"data_compressed_original":243892224,"omap_allocated":0,"internal_metadata":0},"log_size":167261,"ondisk_log_size":167261,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":10222,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542206,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1406956,"kb_used_data":1010724,"kb_used_omap":1,"kb_used_meta":396222,"kb_avail":92964884,"statfs":{"total":96636764160,"available":95196041216,"internally_reserved":0,"allocated":1034981376,"data_stored":1042910983,"data_compressed":31421929,"data_compressed_allocated":32739328,"data_compressed_original":41480192,"omap_allocated":1589,"internal_metadata":405731787},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542206,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406912,"kb_used_data":259188,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":93964928,"statfs":{"total":96636764160,"available":96220086272,"internally_reserved":0,"allocated":265408512,"data_stored":266327003,"data_compressed":7220440,"data_compressed_allocated":7479296,"data_compressed_original":8728576,"omap_allocated":1594,"internal_metadata":151255494},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542206,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1269684,"kb_used_data":891496,"kb_used_omap":1,"kb_used_meta":378174,"kb_avail":93102156,"statfs":{"total":96636764160,"available":95336607744,"internally_reserved":0,"allocated":912891904,"data_stored":917912083,"data_compressed":26020264,"data_compressed_allocated":26992640,"data_compressed_original":32751616,"omap_allocated":1595,"internal_metadata":387250629},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542206,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1904428,"kb_used_data":1372192,"kb_used_omap":1,"kb_used_meta":532222,"kb_avail":92467412,"statfs":{"total":96636764160,"available":94686629888,"internally_reserved":0,"allocated":1405124608,"data_stored":1412894735,"data_compressed":41362870,"data_compressed_allocated":43028480,"data_compressed_original":51744768,"omap_allocated":1595,"internal_metadata":544995781},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542206,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1508044,"kb_used_data":1053504,"kb_used_omap":1,"kb_used_meta":454526,"kb_avail":92863796,"statfs":{"total":96636764160,"available":95092527104,"internally_reserved":0,"allocated":1078788096,"data_stored":1080571807,"data_compressed":21885706,"data_compressed_allocated":22720512,"data_compressed_original":25346048,"omap_allocated":1593,"internal_metadata":465435079},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542206,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":952204,"kb_used_data":673988,"kb_used_omap":1,"kb_used_meta":278206,"kb_avail":93419636,"statfs":{"total":96636764160,"available":95661707264,"internally_reserved":0,"allocated":690163712,"data_stored":694964323,"data_compressed":20980979,"data_compressed_allocated":21929984,"data_compressed_original":27267072,"omap_allocated":1595,"internal_metadata":284883397},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542206,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1461556,"kb_used_data":1079208,"kb_used_omap":1,"kb_used_meta":382334,"kb_avail":92910284,"statfs":{"total":96636764160,"available":95140130816,"internally_reserved":0,"allocated":1105108992,"data_stored":1112879471,"data_compressed":36587350,"data_compressed_allocated":38072320,"data_compressed_original":46706688,"omap_allocated":1595,"internal_metadata":391510469},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574911,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":428420,"kb_used_data":325048,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":93943420,"statfs":{"total":96636764160,"available":96198062080,"internally_reserved":0,"allocated":332849152,"data_stored":334245291,"data_compressed":8816190,"data_compressed_allocated":9150464,"data_compressed_original":10891264,"omap_allocated":1595,"internal_metadata":105839045},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5111,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5111,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1104838656,"data_stored":1112277372,"data_compressed":36582239,"data_compressed_allocated":38039552,"data_compressed_original":46194688,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":689958912,"data_stored":694895920,"data_compressed":20980979,"data_compressed_allocated":21929984,"data_compressed_original":27267072,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":332632064,"data_stored":334172120,"data_compressed":8816190,"data_compressed_allocated":9150464,"data_compressed_original":10891264,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1078571008,"data_stored":1080498636,"data_compressed":21885706,"data_compressed_allocated":22720512,"data_compressed_original":25346048,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1404907520,"data_stored":1412821564,"data_compressed":41362870,"data_compressed_allocated":43028480,"data_compressed_original":51744768,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":912674816,"data_stored":917838912,"data_compressed":26020264,"data_compressed_allocated":26992640,"data_compressed_original":32751616,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":265191424,"data_stored":266253832,"data_compressed":7220440,"data_compressed_allocated":7479296,"data_compressed_original":8728576,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1034711040,"data_stored":1042304932,"data_compressed":31416818,"data_compressed_allocated":32706560,"data_compressed_original":40968192,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:13:21.250 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-05-08T04:08:26.712802+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=26, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.250 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-05-08T04:08:26.712802+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=26, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.250 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-05-08T04:08:26.712802+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=26, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.250 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-05-08T04:08:26.712802+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=26, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.250 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-05-08T04:08:26.712802+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=26, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.250 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-05-08T04:08:26.712802+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=26, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.250 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-05-08T04:08:26.712802+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=26, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.250 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-05-08T04:08:26.712802+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=26, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.251 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-05-08T04:08:26.712802+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=26, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.251 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-05-08T04:08:26.712802+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=26, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.251 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-05-08T04:08:26.712802+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=26, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.251 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-05-08T04:08:26.712802+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=26, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.251 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-05-08T04:08:20.683987+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=20, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.251 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-05-08T04:08:26.712802+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=26, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.251 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-05-08T04:08:20.683987+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=20, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.252 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-05-08T04:08:26.712802+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=26, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.252 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-05-08T04:08:20.683987+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=20, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.252 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-05-08T04:08:26.712802+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=26, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.252 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-05-08T04:08:20.683987+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=20, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.252 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-05-08T04:08:26.712802+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=26, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.252 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-05-08T04:08:26.712802+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=26, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.253 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-05-08T04:08:20.683987+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=20, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.253 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-05-08T04:08:26.712802+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=26, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.253 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-05-08T04:08:20.683987+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=20, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.253 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-05-08T04:08:26.712802+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=26, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.253 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-05-08T04:08:20.683987+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=20, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.253 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-05-08T04:08:26.712802+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=26, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.253 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-05-08T04:08:20.683987+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=20, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.253 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-08T04:08:19.679313+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=19, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.254 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-05-08T04:08:26.712802+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=26, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.254 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-05-08T04:08:26.712802+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=26, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.254 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-05-08T04:08:26.712802+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=26, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.254 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-05-08T04:08:26.712802+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=26, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.254 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-05-08T04:08:26.712802+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=26, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.254 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-05-08T04:08:26.712802+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=26, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.255 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-05-08T04:08:26.712802+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=26, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.255 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-05-08T04:08:26.712802+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=26, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.255 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-05-08T04:08:26.712802+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=26, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.255 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-05-08T04:08:26.712802+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=26, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.255 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-05-08T04:08:26.712802+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=26, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.255 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-05-08T04:08:26.712802+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=8, tm_sec=26, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=4, tm_min=13, tm_sec=16, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T04:13:21.255 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-08T04:13:41.256 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T04:13:41.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.367+0000 7fdd2df17640 1 Processor -- start 2024-05-08T04:13:41.369 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.367+0000 7fdd2df17640 1 -- start start 2024-05-08T04:13:41.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.368+0000 7fdd2df17640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd280735a0 0x7fdd280739e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:41.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.368+0000 7fdd2df17640 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd2806fc90 con 0x7fdd28144110 2024-05-08T04:13:41.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.368+0000 7fdd2df17640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd2806f960 con 0x7fdd2806ee70 2024-05-08T04:13:41.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.368+0000 7fdd2df17640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fdd28144cb0 con 0x7fdd280735a0 2024-05-08T04:13:41.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.368+0000 7fdd27fff640 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fdd28144110 0x7fdd281445a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:46090/0 (socket says 172.21.15.170:46090) 2024-05-08T04:13:41.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.368+0000 7fdd27fff640 1 -- 172.21.15.170:0/1044689220 learned_addr learned my addr 172.21.15.170:0/1044689220 (peer_addr_for_me v1:172.21.15.170:0/0) 2024-05-08T04:13:41.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.368+0000 7fdd1ffff640 1 --2- 172.21.15.170:0/1044689220 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd280735a0 0x7fdd280739e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:41.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.368+0000 7fdd26ffd640 1 -- 172.21.15.170:0/1044689220 <== mon.0 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3951236812 0 0) 0x7fdd2806fc90 con 0x7fdd28144110 2024-05-08T04:13:41.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.368+0000 7fdd26ffd640 1 -- 172.21.15.170:0/1044689220 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd14003520 con 0x7fdd28144110 2024-05-08T04:13:41.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.368+0000 7fdd1ffff640 1 -- 172.21.15.170:0/1044689220 >> v1:172.21.15.187:6789/0 conn(0x7fdd2806ee70 legacy=0x7fdd2806f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:13:41.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.368+0000 7fdd1ffff640 1 -- 172.21.15.170:0/1044689220 >> v1:172.21.15.170:6789/0 conn(0x7fdd28144110 legacy=0x7fdd281445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:41.371 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.368+0000 7fdd1ffff640 1 -- 172.21.15.170:0/1044689220 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd28072dc0 con 0x7fdd280735a0 2024-05-08T04:13:41.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.369+0000 7fdd1ffff640 1 --2- 172.21.15.170:0/1044689220 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd280735a0 0x7fdd280739e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdd100097d0 tx=0x7fdd1002b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e97fe040eebbd770 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:41.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.369+0000 7fdd26ffd640 1 -- 172.21.15.170:0/1044689220 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd10031770 con 0x7fdd280735a0 2024-05-08T04:13:41.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.369+0000 7fdd26ffd640 1 -- 172.21.15.170:0/1044689220 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd10037070 con 0x7fdd280735a0 2024-05-08T04:13:41.372 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.369+0000 7fdd26ffd640 1 -- 172.21.15.170:0/1044689220 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd10031b10 con 0x7fdd280735a0 2024-05-08T04:13:41.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.370+0000 7fdd2df17640 1 -- 172.21.15.170:0/1044689220 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd280735a0 msgr2=0x7fdd280739e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:41.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.370+0000 7fdd2df17640 1 --2- 172.21.15.170:0/1044689220 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd280735a0 0x7fdd280739e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdd100097d0 tx=0x7fdd1002b810 comp rx=0 tx=0).stop 2024-05-08T04:13:41.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.370+0000 7fdd2df17640 1 -- 172.21.15.170:0/1044689220 shutdown_connections 2024-05-08T04:13:41.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.370+0000 7fdd2df17640 1 --2- 172.21.15.170:0/1044689220 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd280735a0 0x7fdd280739e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:41.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.370+0000 7fdd2df17640 1 -- 172.21.15.170:0/1044689220 >> 172.21.15.170:0/1044689220 conn(0x7fdd280b45e0 msgr2=0x7fdd280b49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:41.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.370+0000 7fdd2df17640 1 -- 172.21.15.170:0/1044689220 shutdown_connections 2024-05-08T04:13:41.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.370+0000 7fdd2df17640 1 -- 172.21.15.170:0/1044689220 wait complete. 2024-05-08T04:13:41.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.371+0000 7fdd2df17640 1 Processor -- start 2024-05-08T04:13:41.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.371+0000 7fdd2df17640 1 -- start start 2024-05-08T04:13:41.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.371+0000 7fdd2df17640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd2806ee70 0x7fdd280b0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:41.375 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.371+0000 7fdd2df17640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd280735a0 0x7fdd28137640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:41.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.371+0000 7fdd2df17640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd28144110 0x7fdd28137b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:41.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.371+0000 7fdd2df17640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdd280742c0 con 0x7fdd2806ee70 2024-05-08T04:13:41.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.371+0000 7fdd2df17640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdd28074140 con 0x7fdd28144110 2024-05-08T04:13:41.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.371+0000 7fdd2df17640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7fdd2814f790 con 0x7fdd280735a0 2024-05-08T04:13:41.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.372+0000 7fdd277fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd2806ee70 0x7fdd280b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:41.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.372+0000 7fdd1ffff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd280735a0 0x7fdd28137640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:41.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.372+0000 7fdd277fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd2806ee70 0x7fdd280b0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53920/0 (socket says 172.21.15.170:53920) 2024-05-08T04:13:41.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.372+0000 7fdd277fe640 1 -- 172.21.15.170:0/3655372202 learned_addr learned my addr 172.21.15.170:0/3655372202 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:13:41.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.372+0000 7fdd1ffff640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd280735a0 0x7fdd28137640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:42822/0 (socket says 172.21.15.170:42822) 2024-05-08T04:13:41.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.372+0000 7fdd27fff640 1 --2- 172.21.15.170:0/3655372202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd28144110 0x7fdd28137b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:41.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.372+0000 7fdd277fe640 1 -- 172.21.15.170:0/3655372202 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd280735a0 msgr2=0x7fdd28137640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:41.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.372+0000 7fdd277fe640 1 --2- 172.21.15.170:0/3655372202 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd280735a0 0x7fdd28137640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:41.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.372+0000 7fdd277fe640 1 -- 172.21.15.170:0/3655372202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd28144110 msgr2=0x7fdd28137b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:41.376 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.372+0000 7fdd277fe640 1 --2- 172.21.15.170:0/3655372202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd28144110 0x7fdd28137b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:41.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.372+0000 7fdd277fe640 1 -- 172.21.15.170:0/3655372202 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd280b1450 con 0x7fdd2806ee70 2024-05-08T04:13:41.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.372+0000 7fdd27fff640 1 --2- 172.21.15.170:0/3655372202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd28144110 0x7fdd28137b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T04:13:41.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.372+0000 7fdd1ffff640 1 --2- 172.21.15.170:0/3655372202 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd280735a0 0x7fdd28137640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:13:41.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.372+0000 7fdd277fe640 1 --2- 172.21.15.170:0/3655372202 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd2806ee70 0x7fdd280b0050 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdd1800c3a0 tx=0x7fdd1800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:41.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.372+0000 7fdd24ff9640 1 -- 172.21.15.170:0/3655372202 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd18017070 con 0x7fdd2806ee70 2024-05-08T04:13:41.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.372+0000 7fdd24ff9640 1 -- 172.21.15.170:0/3655372202 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd180052d0 con 0x7fdd2806ee70 2024-05-08T04:13:41.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.372+0000 7fdd2df17640 1 -- 172.21.15.170:0/3655372202 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd280b1730 con 0x7fdd2806ee70 2024-05-08T04:13:41.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.372+0000 7fdd24ff9640 1 -- 172.21.15.170:0/3655372202 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd18003830 con 0x7fdd2806ee70 2024-05-08T04:13:41.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.372+0000 7fdd2df17640 1 -- 172.21.15.170:0/3655372202 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd281380c0 con 0x7fdd2806ee70 2024-05-08T04:13:41.377 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.373+0000 7fdd2df17640 1 -- 172.21.15.170:0/3655372202 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd2806e9e0 con 0x7fdd2806ee70 2024-05-08T04:13:41.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.374+0000 7fdd24ff9640 1 -- 172.21.15.170:0/3655372202 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7fdd1801a070 con 0x7fdd2806ee70 2024-05-08T04:13:41.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.374+0000 7fdd24ff9640 1 --2- 172.21.15.170:0/3655372202 >> v2:172.21.15.187:6816/101007897 conn(0x7fdd08076290 0x7fdd08078750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:41.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.374+0000 7fdd24ff9640 1 -- 172.21.15.170:0/3655372202 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 4525+0+0 (secure 0 0 0) 0x7fdd18096350 con 0x7fdd2806ee70 2024-05-08T04:13:41.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.375+0000 7fdd1ffff640 1 --2- 172.21.15.170:0/3655372202 >> v2:172.21.15.187:6816/101007897 conn(0x7fdd08076290 0x7fdd08078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:41.378 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.375+0000 7fdd1ffff640 1 --2- 172.21.15.170:0/3655372202 >> v2:172.21.15.187:6816/101007897 conn(0x7fdd08076290 0x7fdd08078750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdd10036000 tx=0x7fdd10008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:41.381 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.379+0000 7fdd24ff9640 1 -- 172.21.15.170:0/3655372202 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fdd1805fb90 con 0x7fdd2806ee70 2024-05-08T04:13:41.558 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.556+0000 7fdd2df17640 1 -- 172.21.15.170:0/3655372202 --> v2:172.21.15.187:6816/101007897 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdd28071010 con 0x7fdd08076290 2024-05-08T04:13:41.562 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.560+0000 7fdd24ff9640 1 -- 172.21.15.170:0/3655372202 <== mgr.4107 v2:172.21.15.187:6816/101007897 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+118027 (secure 0 0 0) 0x7fdd28071010 con 0x7fdd08076290 2024-05-08T04:13:41.563 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-05-08T04:13:41.563 INFO:teuthology.orchestra.run.smithi170.stderr:dumped all 2024-05-08T04:13:41.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.562+0000 7fdd2df17640 1 -- 172.21.15.170:0/3655372202 >> v2:172.21.15.187:6816/101007897 conn(0x7fdd08076290 msgr2=0x7fdd08078750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:41.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.562+0000 7fdd2df17640 1 --2- 172.21.15.170:0/3655372202 >> v2:172.21.15.187:6816/101007897 conn(0x7fdd08076290 0x7fdd08078750 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdd10036000 tx=0x7fdd10008030 comp rx=0 tx=0).stop 2024-05-08T04:13:41.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.562+0000 7fdd2df17640 1 -- 172.21.15.170:0/3655372202 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd2806ee70 msgr2=0x7fdd280b0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:41.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.562+0000 7fdd2df17640 1 --2- 172.21.15.170:0/3655372202 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd2806ee70 0x7fdd280b0050 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdd1800c3a0 tx=0x7fdd1800c860 comp rx=0 tx=0).stop 2024-05-08T04:13:41.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.562+0000 7fdd2df17640 1 -- 172.21.15.170:0/3655372202 shutdown_connections 2024-05-08T04:13:41.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.562+0000 7fdd2df17640 1 --2- 172.21.15.170:0/3655372202 >> v2:172.21.15.187:6816/101007897 conn(0x7fdd08076290 0x7fdd08078750 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:41.565 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.562+0000 7fdd2df17640 1 --2- 172.21.15.170:0/3655372202 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd28144110 0x7fdd28137b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:41.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.562+0000 7fdd2df17640 1 --2- 172.21.15.170:0/3655372202 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdd280735a0 0x7fdd28137640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:41.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.562+0000 7fdd2df17640 1 --2- 172.21.15.170:0/3655372202 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd2806ee70 0x7fdd280b0050 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:41.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.562+0000 7fdd2df17640 1 -- 172.21.15.170:0/3655372202 >> 172.21.15.170:0/3655372202 conn(0x7fdd280b45e0 msgr2=0x7fdd28135340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:41.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.562+0000 7fdd2df17640 1 -- 172.21.15.170:0/3655372202 shutdown_connections 2024-05-08T04:13:41.566 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.562+0000 7fdd2df17640 1 -- 172.21.15.170:0/3655372202 wait complete. 2024-05-08T04:13:41.575 INFO:teuthology.orchestra.run.smithi170.stdout:{"pg_ready":true,"pg_map":{"version":177,"stamp":"2024-05-08T04:13:39.724106+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3581797144,"num_objects":882,"num_object_clones":0,"num_object_copies":1764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":882,"num_whiteouts":0,"num_read":2636,"num_read_kb":2641,"num_write":164626,"num_write_kb":5657401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":168213,"ondisk_log_size":168213,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":9325712,"kb_used_data":6665360,"kb_used_omap":12,"kb_used_meta":2660275,"kb_avail":745649008,"statfs":{"total":773094113280,"available":763544584192,"internally_reserved":0,"allocated":6825328640,"data_stored":6862706512,"data_compressed":194295728,"data_compressed_allocated":202113024,"data_compressed_original":244916224,"omap_allocated":12751,"internal_metadata":2724122161},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002450"},"pg_stats":[{"pgid":"3.1f","version":"23'3973","reported_seq":4017,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:22.011378+0000","last_change":"2024-05-08T04:13:22.011427+0000","last_active":"2024-05-08T04:13:22.011378+0000","last_peered":"2024-05-08T04:13:22.011378+0000","last_clean":"2024-05-08T04:13:22.011378+0000","last_became_active":"2024-05-08T04:08:27.731781+0000","last_became_peered":"2024-05-08T04:08:27.731781+0000","last_unstale":"2024-05-08T04:13:22.011378+0000","last_undegraded":"2024-05-08T04:13:22.011378+0000","last_fullsized":"2024-05-08T04:13:22.011378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3973","last_scrub_stamp":"2024-05-08T04:13:22.011312+0000","last_deep_scrub":"23'3973","last_deep_scrub_stamp":"2024-05-08T04:13:22.011312+0000","last_clean_scrub_stamp":"2024-05-08T04:13:22.011312+0000","objects_scrubbed":22,"log_size":3973,"log_dups_size":0,"ondisk_log_size":3973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:45:14.218677+0000","scrub_duration":191,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92274688,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":62,"num_read_kb":52,"num_write":3964,"num_write_kb":169350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"23'7403","reported_seq":7432,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:23.379115+0000","last_change":"2024-05-08T04:13:23.379164+0000","last_active":"2024-05-08T04:13:23.379115+0000","last_peered":"2024-05-08T04:13:23.379115+0000","last_clean":"2024-05-08T04:13:23.379115+0000","last_became_active":"2024-05-08T04:08:27.731598+0000","last_became_peered":"2024-05-08T04:08:27.731598+0000","last_unstale":"2024-05-08T04:13:23.379115+0000","last_undegraded":"2024-05-08T04:13:23.379115+0000","last_fullsized":"2024-05-08T04:13:23.379115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7403","last_scrub_stamp":"2024-05-08T04:13:23.379050+0000","last_deep_scrub":"23'7403","last_deep_scrub_stamp":"2024-05-08T04:13:23.379050+0000","last_clean_scrub_stamp":"2024-05-08T04:13:23.379050+0000","objects_scrubbed":22,"log_size":7403,"log_dups_size":0,"ondisk_log_size":7403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:24:02.689086+0000","scrub_duration":194,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":91594752,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":7365,"num_write_kb":213614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"23'5793","reported_seq":5835,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:25.812527+0000","last_change":"2024-05-08T04:13:25.812527+0000","last_active":"2024-05-08T04:13:25.812527+0000","last_peered":"2024-05-08T04:13:25.812527+0000","last_clean":"2024-05-08T04:13:25.812527+0000","last_became_active":"2024-05-08T04:08:27.732006+0000","last_became_peered":"2024-05-08T04:08:27.732006+0000","last_unstale":"2024-05-08T04:13:25.812527+0000","last_undegraded":"2024-05-08T04:13:25.812527+0000","last_fullsized":"2024-05-08T04:13:25.812527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5793","last_scrub_stamp":"2024-05-08T04:13:25.812429+0000","last_deep_scrub":"23'5793","last_deep_scrub_stamp":"2024-05-08T04:13:25.812429+0000","last_clean_scrub_stamp":"2024-05-08T04:13:25.812429+0000","objects_scrubbed":28,"log_size":5793,"log_dups_size":0,"ondisk_log_size":5793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:21:27.972083+0000","scrub_duration":265,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":56,"num_read_kb":44,"num_write":5770,"num_write_kb":177284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"23'4496","reported_seq":4531,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:26.835500+0000","last_change":"2024-05-08T04:13:26.835500+0000","last_active":"2024-05-08T04:13:26.835500+0000","last_peered":"2024-05-08T04:13:26.835500+0000","last_clean":"2024-05-08T04:13:26.835500+0000","last_became_active":"2024-05-08T04:08:27.732710+0000","last_became_peered":"2024-05-08T04:08:27.732710+0000","last_unstale":"2024-05-08T04:13:26.835500+0000","last_undegraded":"2024-05-08T04:13:26.835500+0000","last_fullsized":"2024-05-08T04:13:26.835500+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4496","last_scrub_stamp":"2024-05-08T04:13:26.835393+0000","last_deep_scrub":"23'4496","last_deep_scrub_stamp":"2024-05-08T04:13:26.835393+0000","last_clean_scrub_stamp":"2024-05-08T04:13:26.835393+0000","objects_scrubbed":30,"log_size":4496,"log_dups_size":0,"ondisk_log_size":4496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:46:14.284432+0000","scrub_duration":267,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125829120,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":4481,"num_write_kb":186066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"23'4205","reported_seq":4235,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:19.093661+0000","last_change":"2024-05-08T04:13:19.093661+0000","last_active":"2024-05-08T04:13:19.093661+0000","last_peered":"2024-05-08T04:13:19.093661+0000","last_clean":"2024-05-08T04:13:19.093661+0000","last_became_active":"2024-05-08T04:08:27.726232+0000","last_became_peered":"2024-05-08T04:08:27.726232+0000","last_unstale":"2024-05-08T04:13:19.093661+0000","last_undegraded":"2024-05-08T04:13:19.093661+0000","last_fullsized":"2024-05-08T04:13:19.093661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4205","last_scrub_stamp":"2024-05-08T04:13:19.093491+0000","last_deep_scrub":"23'4205","last_deep_scrub_stamp":"2024-05-08T04:13:19.093491+0000","last_clean_scrub_stamp":"2024-05-08T04:13:19.093491+0000","objects_scrubbed":30,"log_size":4205,"log_dups_size":0,"ondisk_log_size":4205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:16:26.145147+0000","scrub_duration":239,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120203616,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":4136,"num_write_kb":200298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"23'6168","reported_seq":6199,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:20.846736+0000","last_change":"2024-05-08T04:13:20.846736+0000","last_active":"2024-05-08T04:13:20.846736+0000","last_peered":"2024-05-08T04:13:20.846736+0000","last_clean":"2024-05-08T04:13:20.846736+0000","last_became_active":"2024-05-08T04:08:27.721701+0000","last_became_peered":"2024-05-08T04:08:27.721701+0000","last_unstale":"2024-05-08T04:13:20.846736+0000","last_undegraded":"2024-05-08T04:13:20.846736+0000","last_fullsized":"2024-05-08T04:13:20.846736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6168","last_scrub_stamp":"2024-05-08T04:13:20.846595+0000","last_deep_scrub":"23'6168","last_deep_scrub_stamp":"2024-05-08T04:13:20.846595+0000","last_clean_scrub_stamp":"2024-05-08T04:13:20.846595+0000","objects_scrubbed":27,"log_size":6168,"log_dups_size":0,"ondisk_log_size":6168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:31:12.347150+0000","scrub_duration":240,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113246208,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":6123,"num_write_kb":209934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"23'5222","reported_seq":5253,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:22.387346+0000","last_change":"2024-05-08T04:13:22.387413+0000","last_active":"2024-05-08T04:13:22.387346+0000","last_peered":"2024-05-08T04:13:22.387346+0000","last_clean":"2024-05-08T04:13:22.387346+0000","last_became_active":"2024-05-08T04:08:27.728564+0000","last_became_peered":"2024-05-08T04:08:27.728564+0000","last_unstale":"2024-05-08T04:13:22.387346+0000","last_undegraded":"2024-05-08T04:13:22.387346+0000","last_fullsized":"2024-05-08T04:13:22.387346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5222","last_scrub_stamp":"2024-05-08T04:13:22.387246+0000","last_deep_scrub":"23'5222","last_deep_scrub_stamp":"2024-05-08T04:13:22.387246+0000","last_clean_scrub_stamp":"2024-05-08T04:13:22.387246+0000","objects_scrubbed":35,"log_size":5222,"log_dups_size":0,"ondisk_log_size":5222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:45:37.388028+0000","scrub_duration":216,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142606336,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":5146,"num_write_kb":193669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"23'7594","reported_seq":7626,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:20.548409+0000","last_change":"2024-05-08T04:13:20.548452+0000","last_active":"2024-05-08T04:13:20.548409+0000","last_peered":"2024-05-08T04:13:20.548409+0000","last_clean":"2024-05-08T04:13:20.548409+0000","last_became_active":"2024-05-08T04:08:27.728205+0000","last_became_peered":"2024-05-08T04:08:27.728205+0000","last_unstale":"2024-05-08T04:13:20.548409+0000","last_undegraded":"2024-05-08T04:13:20.548409+0000","last_fullsized":"2024-05-08T04:13:20.548409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7594","last_scrub_stamp":"2024-05-08T04:13:20.548340+0000","last_deep_scrub":"23'7594","last_deep_scrub_stamp":"2024-05-08T04:13:20.548340+0000","last_clean_scrub_stamp":"2024-05-08T04:13:20.548340+0000","objects_scrubbed":34,"log_size":7594,"log_dups_size":0,"ondisk_log_size":7594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:34:24.262347+0000","scrub_duration":257,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142606336,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":12,"num_read_kb":135,"num_write":7569,"num_write_kb":191194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"23'5928","reported_seq":5956,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:23.039832+0000","last_change":"2024-05-08T04:13:23.039872+0000","last_active":"2024-05-08T04:13:23.039832+0000","last_peered":"2024-05-08T04:13:23.039832+0000","last_clean":"2024-05-08T04:13:23.039832+0000","last_became_active":"2024-05-08T04:08:27.731769+0000","last_became_peered":"2024-05-08T04:08:27.731769+0000","last_unstale":"2024-05-08T04:13:23.039832+0000","last_undegraded":"2024-05-08T04:13:23.039832+0000","last_fullsized":"2024-05-08T04:13:23.039832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5928","last_scrub_stamp":"2024-05-08T04:13:23.039759+0000","last_deep_scrub":"23'5928","last_deep_scrub_stamp":"2024-05-08T04:13:23.039759+0000","last_clean_scrub_stamp":"2024-05-08T04:13:23.039759+0000","objects_scrubbed":25,"log_size":5928,"log_dups_size":0,"ondisk_log_size":5928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:01:41.125481+0000","scrub_duration":221,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104857600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5896,"num_write_kb":160193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"23'4640","reported_seq":4669,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:24.815804+0000","last_change":"2024-05-08T04:13:24.815839+0000","last_active":"2024-05-08T04:13:24.815804+0000","last_peered":"2024-05-08T04:13:24.815804+0000","last_clean":"2024-05-08T04:13:24.815804+0000","last_became_active":"2024-05-08T04:08:27.730749+0000","last_became_peered":"2024-05-08T04:08:27.730749+0000","last_unstale":"2024-05-08T04:13:24.815804+0000","last_undegraded":"2024-05-08T04:13:24.815804+0000","last_fullsized":"2024-05-08T04:13:24.815804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4640","last_scrub_stamp":"2024-05-08T04:13:24.815738+0000","last_deep_scrub":"23'4640","last_deep_scrub_stamp":"2024-05-08T04:13:24.815738+0000","last_clean_scrub_stamp":"2024-05-08T04:13:24.815738+0000","objects_scrubbed":32,"log_size":4640,"log_dups_size":0,"ondisk_log_size":4640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:08:20.891232+0000","scrub_duration":288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":127271424,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":4592,"num_write_kb":174343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"23'7496","reported_seq":7835,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:26.200155+0000","last_change":"2024-05-08T04:13:26.200187+0000","last_active":"2024-05-08T04:13:26.200155+0000","last_peered":"2024-05-08T04:13:26.200155+0000","last_clean":"2024-05-08T04:13:26.200155+0000","last_became_active":"2024-05-08T04:08:27.729318+0000","last_became_peered":"2024-05-08T04:08:27.729318+0000","last_unstale":"2024-05-08T04:13:26.200155+0000","last_undegraded":"2024-05-08T04:13:26.200155+0000","last_fullsized":"2024-05-08T04:13:26.200155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7496","last_scrub_stamp":"2024-05-08T04:13:26.200086+0000","last_deep_scrub":"23'7496","last_deep_scrub_stamp":"2024-05-08T04:13:26.200086+0000","last_clean_scrub_stamp":"2024-05-08T04:13:26.200086+0000","objects_scrubbed":29,"log_size":7496,"log_dups_size":0,"ondisk_log_size":7496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:05:17.840296+0000","scrub_duration":241,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112068432,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":1045,"num_read_kb":996,"num_write":7685,"num_write_kb":175598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"23'6117","reported_seq":6145,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:22.888552+0000","last_change":"2024-05-08T04:13:22.888605+0000","last_active":"2024-05-08T04:13:22.888552+0000","last_peered":"2024-05-08T04:13:22.888552+0000","last_clean":"2024-05-08T04:13:22.888552+0000","last_became_active":"2024-05-08T04:08:27.729781+0000","last_became_peered":"2024-05-08T04:08:27.729781+0000","last_unstale":"2024-05-08T04:13:22.888552+0000","last_undegraded":"2024-05-08T04:13:22.888552+0000","last_fullsized":"2024-05-08T04:13:22.888552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6117","last_scrub_stamp":"2024-05-08T04:13:22.888474+0000","last_deep_scrub":"23'6117","last_deep_scrub_stamp":"2024-05-08T04:13:22.888474+0000","last_clean_scrub_stamp":"2024-05-08T04:13:22.888474+0000","objects_scrubbed":30,"log_size":6117,"log_dups_size":0,"ondisk_log_size":6117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:14:52.748271+0000","scrub_duration":307,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123143848,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6035,"num_write_kb":178933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:20.405248+0000","last_change":"2024-05-08T04:13:20.405283+0000","last_active":"2024-05-08T04:13:20.405248+0000","last_peered":"2024-05-08T04:13:20.405248+0000","last_clean":"2024-05-08T04:13:20.405248+0000","last_became_active":"2024-05-08T04:08:21.697149+0000","last_became_peered":"2024-05-08T04:08:21.697149+0000","last_unstale":"2024-05-08T04:13:20.405248+0000","last_undegraded":"2024-05-08T04:13:20.405248+0000","last_fullsized":"2024-05-08T04:13:20.405248+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:13:20.405153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:13:20.405153+0000","last_clean_scrub_stamp":"2024-05-08T04:13:20.405153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:02:40.419111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'7377","reported_seq":7428,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:18.089896+0000","last_change":"2024-05-08T04:13:18.089956+0000","last_active":"2024-05-08T04:13:18.089896+0000","last_peered":"2024-05-08T04:13:18.089896+0000","last_clean":"2024-05-08T04:13:18.089896+0000","last_became_active":"2024-05-08T04:08:27.725165+0000","last_became_peered":"2024-05-08T04:08:27.725165+0000","last_unstale":"2024-05-08T04:13:18.089896+0000","last_undegraded":"2024-05-08T04:13:18.089896+0000","last_fullsized":"2024-05-08T04:13:18.089896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'7377","last_scrub_stamp":"2024-05-08T04:13:18.089829+0000","last_deep_scrub":"23'7377","last_deep_scrub_stamp":"2024-05-08T04:13:18.089829+0000","last_clean_scrub_stamp":"2024-05-08T04:13:18.089829+0000","objects_scrubbed":37,"log_size":7377,"log_dups_size":0,"ondisk_log_size":7377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:39:29.199201+0000","scrub_duration":270,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142738448,"num_objects":37,"num_object_clones":0,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":74,"num_read_kb":59,"num_write":7371,"num_write_kb":187653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:19.136360+0000","last_change":"2024-05-08T04:13:19.136394+0000","last_active":"2024-05-08T04:13:19.136360+0000","last_peered":"2024-05-08T04:13:19.136360+0000","last_clean":"2024-05-08T04:13:19.136360+0000","last_became_active":"2024-05-08T04:08:21.696632+0000","last_became_peered":"2024-05-08T04:08:21.696632+0000","last_unstale":"2024-05-08T04:13:19.136360+0000","last_undegraded":"2024-05-08T04:13:19.136360+0000","last_fullsized":"2024-05-08T04:13:19.136360+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:13:19.136280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:13:19.136280+0000","last_clean_scrub_stamp":"2024-05-08T04:13:19.136280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:17:03.630187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"23'3740","reported_seq":3768,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:22.450837+0000","last_change":"2024-05-08T04:13:22.450865+0000","last_active":"2024-05-08T04:13:22.450837+0000","last_peered":"2024-05-08T04:13:22.450837+0000","last_clean":"2024-05-08T04:13:22.450837+0000","last_became_active":"2024-05-08T04:08:27.730530+0000","last_became_peered":"2024-05-08T04:08:27.730530+0000","last_unstale":"2024-05-08T04:13:22.450837+0000","last_undegraded":"2024-05-08T04:13:22.450837+0000","last_fullsized":"2024-05-08T04:13:22.450837+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3740","last_scrub_stamp":"2024-05-08T04:13:22.450764+0000","last_deep_scrub":"23'3740","last_deep_scrub_stamp":"2024-05-08T04:13:22.450764+0000","last_clean_scrub_stamp":"2024-05-08T04:13:22.450764+0000","objects_scrubbed":28,"log_size":3740,"log_dups_size":0,"ondisk_log_size":3740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:21:16.693257+0000","scrub_duration":225,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117440512,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3661,"num_write_kb":168963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:21.940402+0000","last_change":"2024-05-08T04:13:21.940426+0000","last_active":"2024-05-08T04:13:21.940402+0000","last_peered":"2024-05-08T04:13:21.940402+0000","last_clean":"2024-05-08T04:13:21.940402+0000","last_became_active":"2024-05-08T04:08:21.696403+0000","last_became_peered":"2024-05-08T04:08:21.696403+0000","last_unstale":"2024-05-08T04:13:21.940402+0000","last_undegraded":"2024-05-08T04:13:21.940402+0000","last_fullsized":"2024-05-08T04:13:21.940402+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:13:21.940328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:13:21.940328+0000","last_clean_scrub_stamp":"2024-05-08T04:13:21.940328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:48:09.854833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"23'6960","reported_seq":6988,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:20.295553+0000","last_change":"2024-05-08T04:13:20.295601+0000","last_active":"2024-05-08T04:13:20.295553+0000","last_peered":"2024-05-08T04:13:20.295553+0000","last_clean":"2024-05-08T04:13:20.295553+0000","last_became_active":"2024-05-08T04:08:27.721303+0000","last_became_peered":"2024-05-08T04:08:27.721303+0000","last_unstale":"2024-05-08T04:13:20.295553+0000","last_undegraded":"2024-05-08T04:13:20.295553+0000","last_fullsized":"2024-05-08T04:13:20.295553+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6960","last_scrub_stamp":"2024-05-08T04:13:20.295454+0000","last_deep_scrub":"23'6960","last_deep_scrub_stamp":"2024-05-08T04:13:20.295454+0000","last_clean_scrub_stamp":"2024-05-08T04:13:20.295454+0000","objects_scrubbed":26,"log_size":6960,"log_dups_size":0,"ondisk_log_size":6960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:07:31.066225+0000","scrub_duration":184,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109051904,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6930,"num_write_kb":193544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:18.172118+0000","last_change":"2024-05-08T04:13:18.172137+0000","last_active":"2024-05-08T04:13:18.172118+0000","last_peered":"2024-05-08T04:13:18.172118+0000","last_clean":"2024-05-08T04:13:18.172118+0000","last_became_active":"2024-05-08T04:08:21.693014+0000","last_became_peered":"2024-05-08T04:08:21.693014+0000","last_unstale":"2024-05-08T04:13:18.172118+0000","last_undegraded":"2024-05-08T04:13:18.172118+0000","last_fullsized":"2024-05-08T04:13:18.172118+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:13:18.172046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:13:18.172046+0000","last_clean_scrub_stamp":"2024-05-08T04:13:18.172046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:22:06.050867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"23'4538","reported_seq":4566,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:22.715602+0000","last_change":"2024-05-08T04:13:22.715689+0000","last_active":"2024-05-08T04:13:22.715602+0000","last_peered":"2024-05-08T04:13:22.715602+0000","last_clean":"2024-05-08T04:13:22.715602+0000","last_became_active":"2024-05-08T04:08:27.729504+0000","last_became_peered":"2024-05-08T04:08:27.729504+0000","last_unstale":"2024-05-08T04:13:22.715602+0000","last_undegraded":"2024-05-08T04:13:22.715602+0000","last_fullsized":"2024-05-08T04:13:22.715602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4538","last_scrub_stamp":"2024-05-08T04:13:22.715507+0000","last_deep_scrub":"23'4538","last_deep_scrub_stamp":"2024-05-08T04:13:22.715507+0000","last_clean_scrub_stamp":"2024-05-08T04:13:22.715507+0000","objects_scrubbed":24,"log_size":4538,"log_dups_size":0,"ondisk_log_size":4538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:33:45.633294+0000","scrub_duration":211,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96535048,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":4515,"num_write_kb":160844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"23'6306","reported_seq":6447,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:21.854850+0000","last_change":"2024-05-08T04:13:21.854910+0000","last_active":"2024-05-08T04:13:21.854850+0000","last_peered":"2024-05-08T04:13:21.854850+0000","last_clean":"2024-05-08T04:13:21.854850+0000","last_became_active":"2024-05-08T04:08:27.726527+0000","last_became_peered":"2024-05-08T04:08:27.726527+0000","last_unstale":"2024-05-08T04:13:21.854850+0000","last_undegraded":"2024-05-08T04:13:21.854850+0000","last_fullsized":"2024-05-08T04:13:21.854850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'6306","last_scrub_stamp":"2024-05-08T04:13:21.854783+0000","last_deep_scrub":"23'6306","last_deep_scrub_stamp":"2024-05-08T04:13:21.854783+0000","last_clean_scrub_stamp":"2024-05-08T04:13:21.854783+0000","objects_scrubbed":34,"log_size":6306,"log_dups_size":0,"ondisk_log_size":6306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:45:58.875541+0000","scrub_duration":253,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":126223376,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":153,"num_read_kb":139,"num_write":6308,"num_write_kb":247140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:21.496846+0000","last_change":"2024-05-08T04:13:21.496880+0000","last_active":"2024-05-08T04:13:21.496846+0000","last_peered":"2024-05-08T04:13:21.496846+0000","last_clean":"2024-05-08T04:13:21.496846+0000","last_became_active":"2024-05-08T04:08:21.699013+0000","last_became_peered":"2024-05-08T04:08:21.699013+0000","last_unstale":"2024-05-08T04:13:21.496846+0000","last_undegraded":"2024-05-08T04:13:21.496846+0000","last_fullsized":"2024-05-08T04:13:21.496846+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:13:21.496747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:13:21.496747+0000","last_clean_scrub_stamp":"2024-05-08T04:13:21.496747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:15:43.601047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"23'3092","reported_seq":3119,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:21.265778+0000","last_change":"2024-05-08T04:13:21.265858+0000","last_active":"2024-05-08T04:13:21.265778+0000","last_peered":"2024-05-08T04:13:21.265778+0000","last_clean":"2024-05-08T04:13:21.265778+0000","last_became_active":"2024-05-08T04:08:27.723118+0000","last_became_peered":"2024-05-08T04:08:27.723118+0000","last_unstale":"2024-05-08T04:13:21.265778+0000","last_undegraded":"2024-05-08T04:13:21.265778+0000","last_fullsized":"2024-05-08T04:13:21.265778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3092","last_scrub_stamp":"2024-05-08T04:13:21.265684+0000","last_deep_scrub":"23'3092","last_deep_scrub_stamp":"2024-05-08T04:13:21.265684+0000","last_clean_scrub_stamp":"2024-05-08T04:13:21.265684+0000","objects_scrubbed":23,"log_size":3092,"log_dups_size":0,"ondisk_log_size":3092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:52:39.963059+0000","scrub_duration":121,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3019,"num_write_kb":152195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:19.090619+0000","last_change":"2024-05-08T04:13:19.090640+0000","last_active":"2024-05-08T04:13:19.090619+0000","last_peered":"2024-05-08T04:13:19.090619+0000","last_clean":"2024-05-08T04:13:19.090619+0000","last_became_active":"2024-05-08T04:08:21.798995+0000","last_became_peered":"2024-05-08T04:08:21.798995+0000","last_unstale":"2024-05-08T04:13:19.090619+0000","last_undegraded":"2024-05-08T04:13:19.090619+0000","last_fullsized":"2024-05-08T04:13:19.090619+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:13:19.090547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:13:19.090547+0000","last_clean_scrub_stamp":"2024-05-08T04:13:19.090547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:54:31.737872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"23'3255","reported_seq":3284,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:20.098510+0000","last_change":"2024-05-08T04:13:20.098536+0000","last_active":"2024-05-08T04:13:20.098510+0000","last_peered":"2024-05-08T04:13:20.098510+0000","last_clean":"2024-05-08T04:13:20.098510+0000","last_became_active":"2024-05-08T04:08:27.726457+0000","last_became_peered":"2024-05-08T04:08:27.726457+0000","last_unstale":"2024-05-08T04:13:20.098510+0000","last_undegraded":"2024-05-08T04:13:20.098510+0000","last_fullsized":"2024-05-08T04:13:20.098510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3255","last_scrub_stamp":"2024-05-08T04:13:20.098405+0000","last_deep_scrub":"23'3255","last_deep_scrub_stamp":"2024-05-08T04:13:20.098405+0000","last_clean_scrub_stamp":"2024-05-08T04:13:20.098405+0000","objects_scrubbed":26,"log_size":3255,"log_dups_size":0,"ondisk_log_size":3255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:43:39.143378+0000","scrub_duration":201,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":104923656,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":3211,"num_write_kb":154644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:22.948387+0000","last_change":"2024-05-08T04:13:22.948425+0000","last_active":"2024-05-08T04:13:22.948387+0000","last_peered":"2024-05-08T04:13:22.948387+0000","last_clean":"2024-05-08T04:13:22.948387+0000","last_became_active":"2024-05-08T04:08:21.696370+0000","last_became_peered":"2024-05-08T04:08:21.696370+0000","last_unstale":"2024-05-08T04:13:22.948387+0000","last_undegraded":"2024-05-08T04:13:22.948387+0000","last_fullsized":"2024-05-08T04:13:22.948387+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:13:22.948298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:13:22.948298+0000","last_clean_scrub_stamp":"2024-05-08T04:13:22.948298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:17:17.138080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"23'5094","reported_seq":5121,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:24.347945+0000","last_change":"2024-05-08T04:13:24.347994+0000","last_active":"2024-05-08T04:13:24.347945+0000","last_peered":"2024-05-08T04:13:24.347945+0000","last_clean":"2024-05-08T04:13:24.347945+0000","last_became_active":"2024-05-08T04:08:27.732446+0000","last_became_peered":"2024-05-08T04:08:27.732446+0000","last_unstale":"2024-05-08T04:13:24.347945+0000","last_undegraded":"2024-05-08T04:13:24.347945+0000","last_fullsized":"2024-05-08T04:13:24.347945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5094","last_scrub_stamp":"2024-05-08T04:13:24.347864+0000","last_deep_scrub":"23'5094","last_deep_scrub_stamp":"2024-05-08T04:13:24.347864+0000","last_clean_scrub_stamp":"2024-05-08T04:13:24.347864+0000","objects_scrubbed":20,"log_size":5094,"log_dups_size":0,"ondisk_log_size":5094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:46:15.027936+0000","scrub_duration":168,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83886080,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5070,"num_write_kb":160386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":33,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:20.455113+0000","last_change":"2024-05-08T04:13:20.455148+0000","last_active":"2024-05-08T04:13:20.455113+0000","last_peered":"2024-05-08T04:13:20.455113+0000","last_clean":"2024-05-08T04:13:20.455113+0000","last_became_active":"2024-05-08T04:08:21.696659+0000","last_became_peered":"2024-05-08T04:08:21.696659+0000","last_unstale":"2024-05-08T04:13:20.455113+0000","last_undegraded":"2024-05-08T04:13:20.455113+0000","last_fullsized":"2024-05-08T04:13:20.455113+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T04:13:20.455008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T04:13:20.455008+0000","last_clean_scrub_stamp":"2024-05-08T04:13:20.455008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:59.035481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"23'74","reported_seq":127,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:20.986553+0000","last_change":"2024-05-08T04:13:20.986579+0000","last_active":"2024-05-08T04:13:20.986553+0000","last_peered":"2024-05-08T04:13:20.986553+0000","last_clean":"2024-05-08T04:13:20.986553+0000","last_became_active":"2024-05-08T04:08:20.691204+0000","last_became_peered":"2024-05-08T04:08:20.691204+0000","last_unstale":"2024-05-08T04:13:20.986553+0000","last_undegraded":"2024-05-08T04:13:20.986553+0000","last_fullsized":"2024-05-08T04:13:20.986553+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'74","last_scrub_stamp":"2024-05-08T04:13:20.986484+0000","last_deep_scrub":"23'74","last_deep_scrub_stamp":"2024-05-08T04:13:20.986484+0000","last_clean_scrub_stamp":"2024-05-08T04:13:20.986484+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:26:29.246423+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"23'3967","reported_seq":4000,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:21.399285+0000","last_change":"2024-05-08T04:13:21.399331+0000","last_active":"2024-05-08T04:13:21.399285+0000","last_peered":"2024-05-08T04:13:21.399285+0000","last_clean":"2024-05-08T04:13:21.399285+0000","last_became_active":"2024-05-08T04:08:27.729321+0000","last_became_peered":"2024-05-08T04:08:27.729321+0000","last_unstale":"2024-05-08T04:13:21.399285+0000","last_undegraded":"2024-05-08T04:13:21.399285+0000","last_fullsized":"2024-05-08T04:13:21.399285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3967","last_scrub_stamp":"2024-05-08T04:13:21.399190+0000","last_deep_scrub":"23'3967","last_deep_scrub_stamp":"2024-05-08T04:13:21.399190+0000","last_clean_scrub_stamp":"2024-05-08T04:13:21.399190+0000","objects_scrubbed":27,"log_size":3967,"log_dups_size":0,"ondisk_log_size":3967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:09:57.220260+0000","scrub_duration":155,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109182976,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":22,"num_read_kb":16,"num_write":3879,"num_write_kb":165315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"23'9425","reported_seq":9584,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:19.946489+0000","last_change":"2024-05-08T04:13:19.946534+0000","last_active":"2024-05-08T04:13:19.946489+0000","last_peered":"2024-05-08T04:13:19.946489+0000","last_clean":"2024-05-08T04:13:19.946489+0000","last_became_active":"2024-05-08T04:08:27.721534+0000","last_became_peered":"2024-05-08T04:08:27.721534+0000","last_unstale":"2024-05-08T04:13:19.946489+0000","last_undegraded":"2024-05-08T04:13:19.946489+0000","last_fullsized":"2024-05-08T04:13:19.946489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'9425","last_scrub_stamp":"2024-05-08T04:13:19.946424+0000","last_deep_scrub":"23'9425","last_deep_scrub_stamp":"2024-05-08T04:13:19.946424+0000","last_clean_scrub_stamp":"2024-05-08T04:13:19.946424+0000","objects_scrubbed":35,"log_size":9425,"log_dups_size":0,"ondisk_log_size":9425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T04:58:01.869819+0000","scrub_duration":318,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":139724624,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":407,"num_read_kb":368,"num_write":6462,"num_write_kb":194675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"23'3850","reported_seq":3899,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:21.545220+0000","last_change":"2024-05-08T04:13:21.545304+0000","last_active":"2024-05-08T04:13:21.545220+0000","last_peered":"2024-05-08T04:13:21.545220+0000","last_clean":"2024-05-08T04:13:21.545220+0000","last_became_active":"2024-05-08T04:08:27.725833+0000","last_became_peered":"2024-05-08T04:08:27.725833+0000","last_unstale":"2024-05-08T04:13:21.545220+0000","last_undegraded":"2024-05-08T04:13:21.545220+0000","last_fullsized":"2024-05-08T04:13:21.545220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3850","last_scrub_stamp":"2024-05-08T04:13:21.545142+0000","last_deep_scrub":"23'3850","last_deep_scrub_stamp":"2024-05-08T04:13:21.545142+0000","last_clean_scrub_stamp":"2024-05-08T04:13:21.545142+0000","objects_scrubbed":22,"log_size":3850,"log_dups_size":0,"ondisk_log_size":3850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:33:36.910100+0000","scrub_duration":185,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":85458944,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":91,"num_read_kb":137,"num_write":3837,"num_write_kb":155137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"23'4072","reported_seq":4124,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:19.448309+0000","last_change":"2024-05-08T04:13:19.448345+0000","last_active":"2024-05-08T04:13:19.448309+0000","last_peered":"2024-05-08T04:13:19.448309+0000","last_clean":"2024-05-08T04:13:19.448309+0000","last_became_active":"2024-05-08T04:08:27.727653+0000","last_became_peered":"2024-05-08T04:08:27.727653+0000","last_unstale":"2024-05-08T04:13:19.448309+0000","last_undegraded":"2024-05-08T04:13:19.448309+0000","last_fullsized":"2024-05-08T04:13:19.448309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4072","last_scrub_stamp":"2024-05-08T04:13:19.448242+0000","last_deep_scrub":"23'4072","last_deep_scrub_stamp":"2024-05-08T04:13:19.448242+0000","last_clean_scrub_stamp":"2024-05-08T04:13:19.448242+0000","objects_scrubbed":31,"log_size":4072,"log_dups_size":0,"ondisk_log_size":4072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:16:05.546904+0000","scrub_duration":166,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130023424,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":90,"num_read_kb":141,"num_write":4071,"num_write_kb":155757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"23'5348","reported_seq":5383,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:23.699488+0000","last_change":"2024-05-08T04:13:23.699544+0000","last_active":"2024-05-08T04:13:23.699488+0000","last_peered":"2024-05-08T04:13:23.699488+0000","last_clean":"2024-05-08T04:13:23.699488+0000","last_became_active":"2024-05-08T04:08:27.730569+0000","last_became_peered":"2024-05-08T04:08:27.730569+0000","last_unstale":"2024-05-08T04:13:23.699488+0000","last_undegraded":"2024-05-08T04:13:23.699488+0000","last_fullsized":"2024-05-08T04:13:23.699488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5348","last_scrub_stamp":"2024-05-08T04:13:23.699415+0000","last_deep_scrub":"23'5348","last_deep_scrub_stamp":"2024-05-08T04:13:23.699415+0000","last_clean_scrub_stamp":"2024-05-08T04:13:23.699415+0000","objects_scrubbed":24,"log_size":5348,"log_dups_size":0,"ondisk_log_size":5348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:21:54.510116+0000","scrub_duration":217,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100663296,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":5289,"num_write_kb":148492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"23'4049","reported_seq":4091,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:29.155457+0000","last_change":"2024-05-08T04:13:29.155484+0000","last_active":"2024-05-08T04:13:29.155457+0000","last_peered":"2024-05-08T04:13:29.155457+0000","last_clean":"2024-05-08T04:13:29.155457+0000","last_became_active":"2024-05-08T04:08:27.733070+0000","last_became_peered":"2024-05-08T04:08:27.733070+0000","last_unstale":"2024-05-08T04:13:29.155457+0000","last_undegraded":"2024-05-08T04:13:29.155457+0000","last_fullsized":"2024-05-08T04:13:29.155457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4049","last_scrub_stamp":"2024-05-08T04:13:29.155387+0000","last_deep_scrub":"23'4049","last_deep_scrub_stamp":"2024-05-08T04:13:29.155387+0000","last_clean_scrub_stamp":"2024-05-08T04:13:29.155387+0000","objects_scrubbed":25,"log_size":4049,"log_dups_size":0,"ondisk_log_size":4049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:23:07.450064+0000","scrub_duration":233,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":103481344,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":53,"num_read_kb":105,"num_write":4128,"num_write_kb":186691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"23'3243","reported_seq":3372,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:28.123590+0000","last_change":"2024-05-08T04:13:28.123643+0000","last_active":"2024-05-08T04:13:28.123590+0000","last_peered":"2024-05-08T04:13:28.123590+0000","last_clean":"2024-05-08T04:13:28.123590+0000","last_became_active":"2024-05-08T04:08:27.730372+0000","last_became_peered":"2024-05-08T04:08:27.730372+0000","last_unstale":"2024-05-08T04:13:28.123590+0000","last_undegraded":"2024-05-08T04:13:28.123590+0000","last_fullsized":"2024-05-08T04:13:28.123590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3243","last_scrub_stamp":"2024-05-08T04:13:28.123526+0000","last_deep_scrub":"23'3243","last_deep_scrub_stamp":"2024-05-08T04:13:28.123526+0000","last_clean_scrub_stamp":"2024-05-08T04:13:28.123526+0000","objects_scrubbed":20,"log_size":3243,"log_dups_size":0,"ondisk_log_size":3243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:44:29.877391+0000","scrub_duration":173,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":78381056,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":342,"num_read_kb":267,"num_write":3496,"num_write_kb":113772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"23'4372","reported_seq":4402,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:27.189436+0000","last_change":"2024-05-08T04:13:27.189514+0000","last_active":"2024-05-08T04:13:27.189436+0000","last_peered":"2024-05-08T04:13:27.189436+0000","last_clean":"2024-05-08T04:13:27.189436+0000","last_became_active":"2024-05-08T04:08:27.729584+0000","last_became_peered":"2024-05-08T04:08:27.729584+0000","last_unstale":"2024-05-08T04:13:27.189436+0000","last_undegraded":"2024-05-08T04:13:27.189436+0000","last_fullsized":"2024-05-08T04:13:27.189436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'4372","last_scrub_stamp":"2024-05-08T04:13:27.189353+0000","last_deep_scrub":"23'4372","last_deep_scrub_stamp":"2024-05-08T04:13:27.189353+0000","last_clean_scrub_stamp":"2024-05-08T04:13:27.189353+0000","objects_scrubbed":24,"log_size":4372,"log_dups_size":0,"ondisk_log_size":4372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:12:01.716740+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95482384,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":6,"num_read_kb":5,"num_write":4309,"num_write_kb":147779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"23'5335","reported_seq":5371,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:22.575888+0000","last_change":"2024-05-08T04:13:22.575936+0000","last_active":"2024-05-08T04:13:22.575888+0000","last_peered":"2024-05-08T04:13:22.575888+0000","last_clean":"2024-05-08T04:13:22.575888+0000","last_became_active":"2024-05-08T04:08:27.726692+0000","last_became_peered":"2024-05-08T04:08:27.726692+0000","last_unstale":"2024-05-08T04:13:22.575888+0000","last_undegraded":"2024-05-08T04:13:22.575888+0000","last_fullsized":"2024-05-08T04:13:22.575888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'5335","last_scrub_stamp":"2024-05-08T04:13:22.575819+0000","last_deep_scrub":"23'5335","last_deep_scrub_stamp":"2024-05-08T04:13:22.575819+0000","last_clean_scrub_stamp":"2024-05-08T04:13:22.575819+0000","objects_scrubbed":29,"log_size":5335,"log_dups_size":0,"ondisk_log_size":5335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:47:19.909179+0000","scrub_duration":228,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":120979456,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":5269,"num_write_kb":245378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"23'2467","reported_seq":2494,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:24.117116+0000","last_change":"2024-05-08T04:13:24.117163+0000","last_active":"2024-05-08T04:13:24.117116+0000","last_peered":"2024-05-08T04:13:24.117116+0000","last_clean":"2024-05-08T04:13:24.117116+0000","last_became_active":"2024-05-08T04:08:27.727138+0000","last_became_peered":"2024-05-08T04:08:27.727138+0000","last_unstale":"2024-05-08T04:13:24.117116+0000","last_undegraded":"2024-05-08T04:13:24.117116+0000","last_fullsized":"2024-05-08T04:13:24.117116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'2467","last_scrub_stamp":"2024-05-08T04:13:24.117015+0000","last_deep_scrub":"23'2467","last_deep_scrub_stamp":"2024-05-08T04:13:24.117015+0000","last_clean_scrub_stamp":"2024-05-08T04:13:24.117015+0000","objects_scrubbed":17,"log_size":2467,"log_dups_size":0,"ondisk_log_size":2467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:13:27.292548+0000","scrub_duration":142,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71303168,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2442,"num_write_kb":115332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"23'3814","reported_seq":3849,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:21.039103+0000","last_change":"2024-05-08T04:13:21.039168+0000","last_active":"2024-05-08T04:13:21.039103+0000","last_peered":"2024-05-08T04:13:21.039103+0000","last_clean":"2024-05-08T04:13:21.039103+0000","last_became_active":"2024-05-08T04:08:27.729024+0000","last_became_peered":"2024-05-08T04:08:27.729024+0000","last_unstale":"2024-05-08T04:13:21.039103+0000","last_undegraded":"2024-05-08T04:13:21.039103+0000","last_fullsized":"2024-05-08T04:13:21.039103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'3814","last_scrub_stamp":"2024-05-08T04:13:21.039038+0000","last_deep_scrub":"23'3814","last_deep_scrub_stamp":"2024-05-08T04:13:21.039038+0000","last_clean_scrub_stamp":"2024-05-08T04:13:21.039038+0000","objects_scrubbed":24,"log_size":3814,"log_dups_size":0,"ondisk_log_size":3814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:18:33.068726+0000","scrub_duration":186,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96600064,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":3752,"num_write_kb":156932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"23'8800","reported_seq":8830,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-05-08T04:13:25.365824+0000","last_change":"2024-05-08T04:13:25.365872+0000","last_active":"2024-05-08T04:13:25.365824+0000","last_peered":"2024-05-08T04:13:25.365824+0000","last_clean":"2024-05-08T04:13:25.365824+0000","last_became_active":"2024-05-08T04:08:27.728925+0000","last_became_peered":"2024-05-08T04:08:27.728925+0000","last_unstale":"2024-05-08T04:13:25.365824+0000","last_undegraded":"2024-05-08T04:13:25.365824+0000","last_fullsized":"2024-05-08T04:13:25.365824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"23'8800","last_scrub_stamp":"2024-05-08T04:13:25.365751+0000","last_deep_scrub":"23'8800","last_deep_scrub_stamp":"2024-05-08T04:13:25.365751+0000","last_clean_scrub_stamp":"2024-05-08T04:13:25.365751+0000","objects_scrubbed":40,"log_size":8800,"log_dups_size":0,"ondisk_log_size":8800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:47:54.279111+0000","scrub_duration":432,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163709456,"num_objects":40,"num_object_clones":0,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":8744,"num_write_kb":214668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3581206776,"num_objects":880,"num_object_clones":0,"num_object_copies":1760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":880,"num_whiteouts":0,"num_read":2570,"num_read_kb":2586,"num_write":164525,"num_write_kb":5655773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6823485440,"data_stored":6861063288,"data_compressed":194285506,"data_compressed_allocated":202047488,"data_compressed_original":243892224,"omap_allocated":0,"internal_metadata":0},"log_size":168139,"ondisk_log_size":168139,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":10222,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542210,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1406956,"kb_used_data":1010724,"kb_used_omap":1,"kb_used_meta":396222,"kb_avail":92964884,"statfs":{"total":96636764160,"available":95196041216,"internally_reserved":0,"allocated":1034981376,"data_stored":1042907031,"data_compressed":31421929,"data_compressed_allocated":32739328,"data_compressed_original":41480192,"omap_allocated":1589,"internal_metadata":405731787},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542210,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406908,"kb_used_data":259188,"kb_used_omap":1,"kb_used_meta":147710,"kb_avail":93964932,"statfs":{"total":96636764160,"available":96220090368,"internally_reserved":0,"allocated":265408512,"data_stored":266327003,"data_compressed":7220440,"data_compressed_allocated":7479296,"data_compressed_original":8728576,"omap_allocated":1594,"internal_metadata":151255494},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542210,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1269680,"kb_used_data":891496,"kb_used_omap":1,"kb_used_meta":378174,"kb_avail":93102160,"statfs":{"total":96636764160,"available":95336611840,"internally_reserved":0,"allocated":912891904,"data_stored":917912083,"data_compressed":26020264,"data_compressed_allocated":26992640,"data_compressed_original":32751616,"omap_allocated":1595,"internal_metadata":387250629},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542210,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1904424,"kb_used_data":1372192,"kb_used_omap":1,"kb_used_meta":532222,"kb_avail":92467416,"statfs":{"total":96636764160,"available":94686633984,"internally_reserved":0,"allocated":1405124608,"data_stored":1412894735,"data_compressed":41362870,"data_compressed_allocated":43028480,"data_compressed_original":51744768,"omap_allocated":1595,"internal_metadata":544995781},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542210,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1508040,"kb_used_data":1053504,"kb_used_omap":1,"kb_used_meta":454526,"kb_avail":92863800,"statfs":{"total":96636764160,"available":95092531200,"internally_reserved":0,"allocated":1078788096,"data_stored":1080571807,"data_compressed":21885706,"data_compressed_allocated":22720512,"data_compressed_original":25346048,"omap_allocated":1593,"internal_metadata":465435079},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542210,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":935576,"kb_used_data":674000,"kb_used_omap":1,"kb_used_meta":261566,"kb_avail":93436264,"statfs":{"total":96636764160,"available":95678734336,"internally_reserved":0,"allocated":690176000,"data_stored":694969091,"data_compressed":20980979,"data_compressed_allocated":21929984,"data_compressed_original":27267072,"omap_allocated":1595,"internal_metadata":267844037},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542210,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1465712,"kb_used_data":1079208,"kb_used_omap":1,"kb_used_meta":386494,"kb_avail":92906128,"statfs":{"total":96636764160,"available":95135875072,"internally_reserved":0,"allocated":1105108992,"data_stored":1112879471,"data_compressed":36587350,"data_compressed_allocated":38072320,"data_compressed_original":46706688,"omap_allocated":1595,"internal_metadata":395770309},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574915,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":428416,"kb_used_data":325048,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":93943424,"statfs":{"total":96636764160,"available":96198066176,"internally_reserved":0,"allocated":332849152,"data_stored":334245291,"data_compressed":8816190,"data_compressed_allocated":9150464,"data_compressed_original":10891264,"omap_allocated":1595,"internal_metadata":105839045},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5111,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5111,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1104838656,"data_stored":1112277372,"data_compressed":36582239,"data_compressed_allocated":38039552,"data_compressed_original":46194688,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":689958912,"data_stored":694895920,"data_compressed":20980979,"data_compressed_allocated":21929984,"data_compressed_original":27267072,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":332632064,"data_stored":334172120,"data_compressed":8816190,"data_compressed_allocated":9150464,"data_compressed_original":10891264,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1078571008,"data_stored":1080498636,"data_compressed":21885706,"data_compressed_allocated":22720512,"data_compressed_original":25346048,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1404907520,"data_stored":1412821564,"data_compressed":41362870,"data_compressed_allocated":43028480,"data_compressed_original":51744768,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":912674816,"data_stored":917838912,"data_compressed":26020264,"data_compressed_allocated":26992640,"data_compressed_original":32751616,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":265191424,"data_stored":266253832,"data_compressed":7220440,"data_compressed_allocated":7479296,"data_compressed_original":8728576,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1034711040,"data_stored":1042304932,"data_compressed":31416818,"data_compressed_allocated":32706560,"data_compressed_original":40968192,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T04:13:41.578 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-08T04:13:41.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.675+0000 7f395d69c640 1 Processor -- start 2024-05-08T04:13:41.677 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.675+0000 7f395d69c640 1 -- start start 2024-05-08T04:13:41.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.676+0000 7f395d69c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f395806d870 0x7f3958140f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:41.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.676+0000 7f395d69c640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3958141440 0x7f3958145480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:41.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.676+0000 7f395d69c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f39581459c0 0x7f3958072e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:41.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.676+0000 7f395d69c640 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 -- 0x7f395806dc50 con 0x7f3958141440 2024-05-08T04:13:41.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.676+0000 7f395d69c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3958141880 con 0x7f39581459c0 2024-05-08T04:13:41.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.676+0000 7f395d69c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f39580ab290 con 0x7f395806d870 2024-05-08T04:13:41.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.676+0000 7f39567fc640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3958141440 0x7f3958145480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:41.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.676+0000 7f3956ffd640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f395806d870 0x7f3958140f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:41.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.676+0000 7f39567fc640 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3958141440 0x7f3958145480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53922/0 (socket says 172.21.15.170:53922) 2024-05-08T04:13:41.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.676+0000 7f3956ffd640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f395806d870 0x7f3958140f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:42838/0 (socket says 172.21.15.170:42838) 2024-05-08T04:13:41.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.676+0000 7f39567fc640 1 -- 172.21.15.170:0/1311441536 learned_addr learned my addr 172.21.15.170:0/1311441536 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:13:41.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.676+0000 7f39577fe640 1 --2- 172.21.15.170:0/1311441536 >> v2:172.21.15.187:3300/0 conn(0x7f39581459c0 0x7f3958072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:41.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.677+0000 7f3956ffd640 1 -- 172.21.15.170:0/1311441536 >> v2:172.21.15.187:3300/0 conn(0x7f39581459c0 msgr2=0x7f3958072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:13:41.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.677+0000 7f3956ffd640 1 --2- 172.21.15.170:0/1311441536 >> v2:172.21.15.187:3300/0 conn(0x7f39581459c0 0x7f3958072e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:41.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.677+0000 7f3956ffd640 1 -- 172.21.15.170:0/1311441536 >> v2:172.21.15.170:3300/0 conn(0x7f3958141440 msgr2=0x7f3958145480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T04:13:41.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.677+0000 7f3956ffd640 1 --2- 172.21.15.170:0/1311441536 >> v2:172.21.15.170:3300/0 conn(0x7f3958141440 0x7f3958145480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:41.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.677+0000 7f3956ffd640 1 -- 172.21.15.170:0/1311441536 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39580aaa80 con 0x7f395806d870 2024-05-08T04:13:41.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.677+0000 7f39567fc640 1 --2- 172.21.15.170:0/1311441536 >> v2:172.21.15.170:3300/0 conn(0x7f3958141440 0x7f3958145480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T04:13:41.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.677+0000 7f3956ffd640 1 --2- 172.21.15.170:0/1311441536 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f395806d870 0x7f3958140f00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f395806fbb0 tx=0x7f394802bd00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b38f7382cf96323e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:41.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.677+0000 7f3955ffb640 1 -- 172.21.15.170:0/1311441536 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3948039070 con 0x7f395806d870 2024-05-08T04:13:41.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.677+0000 7f3955ffb640 1 -- 172.21.15.170:0/1311441536 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f394802cab0 con 0x7f395806d870 2024-05-08T04:13:41.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.677+0000 7f3955ffb640 1 -- 172.21.15.170:0/1311441536 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3948034440 con 0x7f395806d870 2024-05-08T04:13:41.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.678+0000 7f395d69c640 1 -- 172.21.15.170:0/1311441536 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f395806d870 msgr2=0x7f3958140f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:41.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.678+0000 7f395d69c640 1 --2- 172.21.15.170:0/1311441536 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f395806d870 0x7f3958140f00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f395806fbb0 tx=0x7f394802bd00 comp rx=0 tx=0).stop 2024-05-08T04:13:41.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.678+0000 7f395d69c640 1 -- 172.21.15.170:0/1311441536 shutdown_connections 2024-05-08T04:13:41.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.678+0000 7f395d69c640 1 --2- 172.21.15.170:0/1311441536 >> v2:172.21.15.170:3300/0 conn(0x7f3958141440 0x7f3958145480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:41.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.678+0000 7f395d69c640 1 --2- 172.21.15.170:0/1311441536 >> v2:172.21.15.187:3300/0 conn(0x7f39581459c0 0x7f3958072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:41.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.678+0000 7f395d69c640 1 --2- 172.21.15.170:0/1311441536 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f395806d870 0x7f3958140f00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:41.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.678+0000 7f395d69c640 1 -- 172.21.15.170:0/1311441536 >> 172.21.15.170:0/1311441536 conn(0x7f395806ee70 msgr2=0x7f395806f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:41.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.678+0000 7f395d69c640 1 -- 172.21.15.170:0/1311441536 shutdown_connections 2024-05-08T04:13:41.682 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.678+0000 7f395d69c640 1 -- 172.21.15.170:0/1311441536 wait complete. 2024-05-08T04:13:41.684 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.679+0000 7f395d69c640 1 Processor -- start 2024-05-08T04:13:41.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.679+0000 7f395d69c640 1 -- start start 2024-05-08T04:13:41.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.680+0000 7f395d69c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f395806d870 0x7f39581ce350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:41.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.680+0000 7f395d69c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3958141440 0x7f39581ce890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:41.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.680+0000 7f395d69c640 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f39581459c0 0x7f39581cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:41.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.680+0000 7f395d69c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f39580ab950 con 0x7f395806d870 2024-05-08T04:13:41.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.680+0000 7f395d69c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f39580ab7d0 con 0x7f3958141440 2024-05-08T04:13:41.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.680+0000 7f395d69c640 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 -- 0x7f3958145e30 con 0x7f39581459c0 2024-05-08T04:13:41.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.680+0000 7f3956ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f395806d870 0x7f39581ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:41.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.680+0000 7f3956ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f395806d870 0x7f39581ce350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53934/0 (socket says 172.21.15.170:53934) 2024-05-08T04:13:41.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.680+0000 7f3956ffd640 1 -- 172.21.15.170:0/1655183245 learned_addr learned my addr 172.21.15.170:0/1655183245 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-05-08T04:13:41.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.680+0000 7f39577fe640 1 --2- 172.21.15.170:0/1655183245 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f39581459c0 0x7f39581cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:41.685 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.680+0000 7f3956ffd640 1 -- 172.21.15.170:0/1655183245 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f39581459c0 msgr2=0x7f39581cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:41.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.680+0000 7f3956ffd640 1 --2- 172.21.15.170:0/1655183245 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f39581459c0 0x7f39581cb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:41.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.680+0000 7f3956ffd640 1 -- 172.21.15.170:0/1655183245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3958141440 msgr2=0x7f39581ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:41.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.680+0000 7f3956ffd640 1 --2- 172.21.15.170:0/1655183245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3958141440 0x7f39581ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:41.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.680+0000 7f3956ffd640 1 -- 172.21.15.170:0/1655183245 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39581cc6e0 con 0x7f395806d870 2024-05-08T04:13:41.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.680+0000 7f3956ffd640 1 --2- 172.21.15.170:0/1655183245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f395806d870 0x7f39581ce350 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3948037040 tx=0x7f3948004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:41.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.680+0000 7f394f7fe640 1 -- 172.21.15.170:0/1655183245 <== mon.0 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3948039050 con 0x7f395806d870 2024-05-08T04:13:41.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.680+0000 7f394f7fe640 1 -- 172.21.15.170:0/1655183245 <== mon.0 v2:172.21.15.170:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3948045070 con 0x7f395806d870 2024-05-08T04:13:41.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.681+0000 7f394f7fe640 1 -- 172.21.15.170:0/1655183245 <== mon.0 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394803e850 con 0x7f395806d870 2024-05-08T04:13:41.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.681+0000 7f395d69c640 1 -- 172.21.15.170:0/1655183245 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39581cd8e0 con 0x7f395806d870 2024-05-08T04:13:41.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.681+0000 7f395d69c640 1 -- 172.21.15.170:0/1655183245 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39580b2b90 con 0x7f395806d870 2024-05-08T04:13:41.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.681+0000 7f395d69c640 1 -- 172.21.15.170:0/1655183245 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f395806e9e0 con 0x7f395806d870 2024-05-08T04:13:41.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.682+0000 7f394f7fe640 1 -- 172.21.15.170:0/1655183245 <== mon.0 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 6) ==== 98166+0+0 (secure 0 0 0) 0x7f394803e9f0 con 0x7f395806d870 2024-05-08T04:13:41.686 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.682+0000 7f394f7fe640 1 --2- 172.21.15.170:0/1655183245 >> v2:172.21.15.187:6816/101007897 conn(0x7f3938076290 0x7f3938078750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T04:13:41.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.683+0000 7f394f7fe640 1 -- 172.21.15.170:0/1655183245 <== mon.0 v2:172.21.15.170:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 4525+0+0 (secure 0 0 0) 0x7f39480b9e00 con 0x7f395806d870 2024-05-08T04:13:41.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.683+0000 7f39567fc640 1 --2- 172.21.15.170:0/1655183245 >> v2:172.21.15.187:6816/101007897 conn(0x7f3938076290 0x7f3938078750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T04:13:41.687 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.683+0000 7f39567fc640 1 --2- 172.21.15.170:0/1655183245 >> v2:172.21.15.187:6816/101007897 conn(0x7f3938076290 0x7f3938078750 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3940009630 tx=0x7f3940006e90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T04:13:41.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.687+0000 7f394f7fe640 1 -- 172.21.15.170:0/1655183245 <== mon.0 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f39580b2b90 con 0x7f395806d870 2024-05-08T04:13:41.870 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.869+0000 7f395d69c640 1 -- 172.21.15.170:0/1655183245 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f395806e4f0 con 0x7f395806d870 2024-05-08T04:13:41.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.872+0000 7f394f7fe640 1 -- 172.21.15.170:0/1655183245 <== mon.0 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v6)=0 v6) ==== 123+0+0 (secure 0 0 0) 0x7f3948083c90 con 0x7f395806d870 2024-05-08T04:13:41.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.873+0000 7f394f7fe640 1 -- 172.21.15.170:0/1655183245 <== mon.0 v2:172.21.15.170:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f394803a040 con 0x7f395806d870 2024-05-08T04:13:41.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.875+0000 7f395d69c640 1 -- 172.21.15.170:0/1655183245 >> v2:172.21.15.187:6816/101007897 conn(0x7f3938076290 msgr2=0x7f3938078750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:41.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.875+0000 7f395d69c640 1 --2- 172.21.15.170:0/1655183245 >> v2:172.21.15.187:6816/101007897 conn(0x7f3938076290 0x7f3938078750 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3940009630 tx=0x7f3940006e90 comp rx=0 tx=0).stop 2024-05-08T04:13:41.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.875+0000 7f395d69c640 1 -- 172.21.15.170:0/1655183245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f395806d870 msgr2=0x7f39581ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T04:13:41.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.875+0000 7f395d69c640 1 --2- 172.21.15.170:0/1655183245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f395806d870 0x7f39581ce350 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3948037040 tx=0x7f3948004a40 comp rx=0 tx=0).stop 2024-05-08T04:13:41.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.875+0000 7f395d69c640 1 -- 172.21.15.170:0/1655183245 shutdown_connections 2024-05-08T04:13:41.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.875+0000 7f395d69c640 1 --2- 172.21.15.170:0/1655183245 >> v2:172.21.15.187:6816/101007897 conn(0x7f3938076290 0x7f3938078750 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:41.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.875+0000 7f395d69c640 1 --2- 172.21.15.170:0/1655183245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3958141440 0x7f39581ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:41.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.875+0000 7f395d69c640 1 --2- 172.21.15.170:0/1655183245 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f39581459c0 0x7f39581cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:41.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.875+0000 7f395d69c640 1 --2- 172.21.15.170:0/1655183245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f395806d870 0x7f39581ce350 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T04:13:41.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.875+0000 7f395d69c640 1 -- 172.21.15.170:0/1655183245 >> 172.21.15.170:0/1655183245 conn(0x7f395806ee70 msgr2=0x7f3958142660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T04:13:41.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.875+0000 7f395d69c640 1 -- 172.21.15.170:0/1655183245 shutdown_connections 2024-05-08T04:13:41.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-05-08T04:13:41.875+0000 7f395d69c640 1 -- 172.21.15.170:0/1655183245 wait complete. 2024-05-08T04:13:41.888 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-08T04:13:41.888 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-08T04:13:41.888 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-08T04:13:41.888 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:13:41.941 INFO:tasks.ceph.osd.0:Stopped 2024-05-08T04:13:41.942 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-08T04:13:41.942 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:13:41.985 INFO:tasks.ceph.osd.1:Stopped 2024-05-08T04:13:41.986 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-08T04:13:41.986 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:13:42.030 INFO:tasks.ceph.osd.2:Stopped 2024-05-08T04:13:42.030 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-08T04:13:42.030 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:13:42.087 INFO:tasks.ceph.osd.3:Stopped 2024-05-08T04:13:42.088 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-08T04:13:42.088 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:13:42.150 INFO:tasks.ceph.osd.4:Stopped 2024-05-08T04:13:42.150 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-08T04:13:42.150 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:13:42.201 INFO:tasks.ceph.osd.5:Stopped 2024-05-08T04:13:42.201 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-08T04:13:42.201 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:13:42.241 INFO:tasks.ceph.osd.6:Stopped 2024-05-08T04:13:42.241 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-08T04:13:42.241 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:13:42.328 INFO:tasks.ceph.osd.7:Stopped 2024-05-08T04:13:42.328 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-08T04:13:42.328 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-08T04:13:42.328 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:13:42.353 INFO:tasks.ceph.mgr.y:Stopped 2024-05-08T04:13:42.353 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-08T04:13:42.353 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:13:42.381 INFO:tasks.ceph.mgr.x:Stopped 2024-05-08T04:13:42.381 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-08T04:13:42.381 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-08T04:13:42.381 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:13:42.405 INFO:tasks.ceph.mon.a:Stopped 2024-05-08T04:13:42.405 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-08T04:13:42.405 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:13:42.455 INFO:tasks.ceph.mon.c:Stopped 2024-05-08T04:13:42.456 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-08T04:13:42.456 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T04:13:42.467 INFO:tasks.ceph.mon.b:Stopped 2024-05-08T04:13:42.467 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-08T04:13:42.467 DEBUG:teuthology.orchestra.run.smithi170:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | head -n 1 2024-05-08T04:13:42.531 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi170.front.sepia.ceph.com 2024-05-08T04:13:42.531 DEBUG:teuthology.orchestra.run.smithi170:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-08T04:13:43.534 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi170.front.sepia.ceph.com 2024-05-08T04:13:43.534 DEBUG:teuthology.orchestra.run.smithi170:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-08T04:13:43.661 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi170.front.sepia.ceph.com 2024-05-08T04:13:43.661 DEBUG:teuthology.orchestra.run.smithi170:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-08T04:13:43.719 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi170.front.sepia.ceph.com 2024-05-08T04:13:43.719 DEBUG:teuthology.orchestra.run.smithi170:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-08T04:13:43.812 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi187.front.sepia.ceph.com 2024-05-08T04:13:43.812 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-08T04:13:44.096 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi187.front.sepia.ceph.com 2024-05-08T04:13:44.096 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-08T04:13:44.189 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi187.front.sepia.ceph.com 2024-05-08T04:13:44.189 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-08T04:13:44.243 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi187.front.sepia.ceph.com 2024-05-08T04:13:44.243 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-08T04:13:44.306 INFO:tasks.ceph:Archiving mon data... 2024-05-08T04:13:44.307 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/kchai-2024-05-08_00:43:58-rados-wip-kefu-testing-2024-05-06-distro-default-smithi/7696821/data/mon.a.tgz 2024-05-08T04:13:44.307 DEBUG:teuthology.orchestra.run.smithi170:> mktemp 2024-05-08T04:13:44.324 INFO:teuthology.orchestra.run.smithi170.stdout:/tmp/tmp.eLcnncy6g6 2024-05-08T04:13:44.324 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.eLcnncy6g6 2024-05-08T04:13:44.450 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0666 /tmp/tmp.eLcnncy6g6 2024-05-08T04:13:44.534 DEBUG:teuthology.orchestra.remote:smithi170:/tmp/tmp.eLcnncy6g6 is 293KB 2024-05-08T04:13:44.629 DEBUG:teuthology.orchestra.run.smithi170:> rm -fr /tmp/tmp.eLcnncy6g6 2024-05-08T04:13:44.644 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/kchai-2024-05-08_00:43:58-rados-wip-kefu-testing-2024-05-06-distro-default-smithi/7696821/data/mon.c.tgz 2024-05-08T04:13:44.644 DEBUG:teuthology.orchestra.run.smithi170:> mktemp 2024-05-08T04:13:44.698 INFO:teuthology.orchestra.run.smithi170.stdout:/tmp/tmp.lTbNtEQ14z 2024-05-08T04:13:44.699 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.lTbNtEQ14z 2024-05-08T04:13:44.812 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0666 /tmp/tmp.lTbNtEQ14z 2024-05-08T04:13:44.893 DEBUG:teuthology.orchestra.remote:smithi170:/tmp/tmp.lTbNtEQ14z is 293KB 2024-05-08T04:13:44.988 DEBUG:teuthology.orchestra.run.smithi170:> rm -fr /tmp/tmp.lTbNtEQ14z 2024-05-08T04:13:45.003 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/kchai-2024-05-08_00:43:58-rados-wip-kefu-testing-2024-05-06-distro-default-smithi/7696821/data/mon.b.tgz 2024-05-08T04:13:45.003 DEBUG:teuthology.orchestra.run.smithi187:> mktemp 2024-05-08T04:13:45.020 INFO:teuthology.orchestra.run.smithi187.stdout:/tmp/tmp.kukG5ULuYZ 2024-05-08T04:13:45.020 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.kukG5ULuYZ 2024-05-08T04:13:45.151 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0666 /tmp/tmp.kukG5ULuYZ 2024-05-08T04:13:45.249 DEBUG:teuthology.orchestra.remote:smithi187:/tmp/tmp.kukG5ULuYZ is 293KB 2024-05-08T04:13:45.344 DEBUG:teuthology.orchestra.run.smithi187:> rm -fr /tmp/tmp.kukG5ULuYZ 2024-05-08T04:13:45.359 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-08T04:13:45.359 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-08T04:13:45.361 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-08T04:13:45.440 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-08T04:13:45.477 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-08T04:13:45.477 INFO:tasks.ceph:Archiving crash dumps... 2024-05-08T04:13:45.479 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/lib/ceph/crash to /home/teuthworker/archive/kchai-2024-05-08_00:43:58-rados-wip-kefu-testing-2024-05-06-distro-default-smithi/7696821/remote/smithi170/crash 2024-05-08T04:13:45.480 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-08T04:13:45.504 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/lib/ceph/crash to /home/teuthworker/archive/kchai-2024-05-08_00:43:58-rados-wip-kefu-testing-2024-05-06-distro-default-smithi/7696821/remote/smithi187/crash 2024-05-08T04:13:45.505 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-08T04:13:45.541 INFO:tasks.ceph:Compressing logs... 2024-05-08T04:13:45.541 DEBUG:teuthology.orchestra.run.smithi170:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-08T04:13:45.549 DEBUG:teuthology.orchestra.run.smithi187:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-08T04:13:45.585 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34404.log 2024-05-08T04:13:45.585 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.36107.log 2024-05-08T04:13:45.585 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35370.log 2024-05-08T04:13:45.585 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.libcephsqlite.36107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36575.log 2024-05-08T04:13:45.585 INFO:teuthology.orchestra.run.smithi170.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34404.log.gz 2024-05-08T04:13:45.585 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.35370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34411.log 2024-05-08T04:13:45.585 INFO:teuthology.orchestra.run.smithi170.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35370.log.gz 2024-05-08T04:13:45.585 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34754.log 2024-05-08T04:13:45.585 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34750.log 2024-05-08T04:13:45.585 INFO:teuthology.orchestra.run.smithi170.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36575.log.gz 2024-05-08T04:13:45.585 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34754.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34411.log.gz 2024-05-08T04:13:45.586 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34744.log 2024-05-08T04:13:45.586 INFO:teuthology.orchestra.run.smithi170.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34754.log.gz 2024-05-08T04:13:45.586 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36701.log 2024-05-08T04:13:45.586 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34744.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-08T04:13:45.586 INFO:teuthology.orchestra.run.smithi170.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34750.log.gz 2024-05-08T04:13:45.586 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36701.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34744.log.gz 2024-05-08T04:13:45.586 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36785.log 2024-05-08T04:13:45.586 INFO:teuthology.orchestra.run.smithi170.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36701.log.gz 2024-05-08T04:13:45.586 INFO:teuthology.orchestra.run.smithi170.stderr:gzip/var/log/ceph/ceph.log: -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.36064.log 2024-05-08T04:13:45.586 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36785.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-08T04:13:45.586 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.libcephsqlite.36064.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36785.log.gz 2024-05-08T04:13:45.586 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34194.log 2024-05-08T04:13:45.586 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36743.log 2024-05-08T04:13:45.586 INFO:teuthology.orchestra.run.smithi170.stderr: 92.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-08T04:13:45.587 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34970.log 2024-05-08T04:13:45.587 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36743.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34194.log.gz 2024-05-08T04:13:45.587 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34747.log 2024-05-08T04:13:45.587 INFO:teuthology.orchestra.run.smithi170.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36743.log.gz 2024-05-08T04:13:45.587 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34970.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-08T04:13:45.587 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34417.log 2024-05-08T04:13:45.587 INFO:teuthology.orchestra.run.smithi170.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34970.log.gz 2024-05-08T04:13:45.587 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34747.log: /var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34742.log 2024-05-08T04:13:45.587 INFO:teuthology.orchestra.run.smithi170.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34747.log.gz 2024-05-08T04:13:45.587 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34417.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-05-08T04:13:45.587 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30622.log 2024-05-08T04:13:45.587 INFO:teuthology.orchestra.run.smithi170.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34417.log.gz 2024-05-08T04:13:45.587 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-osd.3.log: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-08T04:13:45.587 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34418.log 2024-05-08T04:13:45.587 INFO:teuthology.orchestra.run.smithi170.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34742.log.gz 2024-05-08T04:13:45.588 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36869.log 2024-05-08T04:13:45.588 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.30622.log: /var/log/ceph/ceph-client.admin.34418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36323.log 2024-05-08T04:13:45.588 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.30622.log.gz 2024-05-08T04:13:45.588 INFO:teuthology.orchestra.run.smithi170.stderr: 89.0%/var/log/ceph/ceph-client.admin.36869.log: -- replaced with /var/log/ceph/ceph-client.admin.34418.log.gz 2024-05-08T04:13:45.588 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37163.log 2024-05-08T04:13:45.588 INFO:teuthology.orchestra.run.smithi170.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36869.log.gz 2024-05-08T04:13:45.588 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35084.log 2024-05-08T04:13:45.588 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36323.log.gz 2024-05-08T04:13:45.588 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36911.log 2024-05-08T04:13:45.588 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37163.log: /var/log/ceph/ceph-client.admin.35084.log: gzip 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37163.log.gz 2024-05-08T04:13:45.588 INFO:teuthology.orchestra.run.smithi170.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-05-08T04:13:45.588 INFO:teuthology.orchestra.run.smithi170.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35084.log.gz 2024-05-08T04:13:45.588 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36911.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-05-08T04:13:45.588 INFO:teuthology.orchestra.run.smithi170.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36911.log.gz 2024-05-08T04:13:45.588 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.35493.log 2024-05-08T04:13:45.589 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.36009.log 2024-05-08T04:13:45.589 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.libcephsqlite.35493.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-08T04:13:45.589 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.libcephsqlite.36009.log: 94.5% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.36064.log.gz 2024-05-08T04:13:45.589 INFO:teuthology.orchestra.run.smithi170.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.36009.log.gz 2024-05-08T04:13:45.589 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36365.log 2024-05-08T04:13:45.589 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35087.log 2024-05-08T04:13:45.589 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36365.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36365.log.gz 2024-05-08T04:13:45.589 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34414.log 2024-05-08T04:13:45.589 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.35087.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35087.log.gz 2024-05-08T04:13:45.589 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30308.log 2024-05-08T04:13:45.595 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34414.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34414.log.gz 2024-05-08T04:13:45.595 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34415.log 2024-05-08T04:13:45.595 INFO:teuthology.orchestra.run.smithi170.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-05-08T04:13:45.595 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.30308.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.30308.log.gz 2024-05-08T04:13:45.595 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34152.log 2024-05-08T04:13:45.595 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36617.log 2024-05-08T04:13:45.595 INFO:teuthology.orchestra.run.smithi170.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34415.log.gz 2024-05-08T04:13:45.596 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.35967.log 2024-05-08T04:13:45.596 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36617.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34152.log.gz 2024-05-08T04:13:45.596 INFO:teuthology.orchestra.run.smithi170.stderr:gzip 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36617.log.gz 2024-05-08T04:13:45.596 INFO:teuthology.orchestra.run.smithi170.stderr: -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-05-08T04:13:45.596 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35328.log 2024-05-08T04:13:45.596 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-client.libcephsqlite.35967.log: 91.5% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.35967.log.gz 2024-05-08T04:13:45.596 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-08T04:13:45.596 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.35328.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35328.log.gz 2024-05-08T04:13:45.596 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37245.log 2024-05-08T04:13:45.605 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35286.log 2024-05-08T04:13:45.605 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37245.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37245.log.gz 2024-05-08T04:13:45.608 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30056.log 2024-05-08T04:13:45.608 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29925.log 2024-05-08T04:13:45.608 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-08T04:13:45.608 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.30056.log: /var/log/ceph/ceph-client.admin.29925.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-08T04:13:45.608 INFO:teuthology.orchestra.run.smithi187.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30056.log.gz 2024-05-08T04:13:45.609 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.29925.log.gz 2024-05-08T04:13:45.609 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-05-08T04:13:45.609 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-05-08T04:13:45.609 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-05-08T04:13:45.609 INFO:teuthology.orchestra.run.smithi187.stderr: 85.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-08T04:13:45.609 INFO:teuthology.orchestra.run.smithi187.stderr: 89.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-08T04:13:45.609 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-08T04:13:45.609 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30096.log 2024-05-08T04:13:45.613 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-05-08T04:13:45.613 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.30096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30176.log 2024-05-08T04:13:45.613 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-osd.6.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.30096.log.gz 2024-05-08T04:13:45.613 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30136.log 2024-05-08T04:13:45.613 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.30176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30007.log 2024-05-08T04:13:45.613 INFO:teuthology.orchestra.run.smithi187.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.30176.log.gz 2024-05-08T04:13:45.613 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.30136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29965.log 2024-05-08T04:13:45.614 INFO:teuthology.orchestra.run.smithi187.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.30136.log.gz 2024-05-08T04:13:45.614 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.30007.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-05-08T04:13:45.614 INFO:teuthology.orchestra.run.smithi187.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.30007.log.gz 2024-05-08T04:13:45.614 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30216.log 2024-05-08T04:13:45.614 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.29965.log: 88.3%/var/log/ceph/ceph-client.admin.30216.log: -- replaced with /var/log/ceph/ceph-client.admin.29965.log.gz 2024-05-08T04:13:45.614 INFO:teuthology.orchestra.run.smithi187.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.30216.log.gz 2024-05-08T04:13:45.615 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30725.log 2024-05-08T04:13:45.615 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.35286.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35286.log.gz 2024-05-08T04:13:45.623 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.35988.log 2024-05-08T04:13:45.624 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.30725.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.30725.log.gz 2024-05-08T04:13:45.639 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.36133.log 2024-05-08T04:13:45.640 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.libcephsqlite.35988.log: 90.0% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.35988.log.gz 2024-05-08T04:13:45.648 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37079.log 2024-05-08T04:13:45.650 INFO:teuthology.orchestra.run.smithi187.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-08T04:13:45.667 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.libcephsqlite.36133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34416.log 2024-05-08T04:13:45.668 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37079.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37079.log.gz 2024-05-08T04:13:45.675 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.36152.log 2024-05-08T04:13:45.684 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37121.log 2024-05-08T04:13:45.687 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.libcephsqlite.36152.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.34416.log.gz 2024-05-08T04:13:45.697 INFO:teuthology.orchestra.run.smithi170.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.36152.log.gz 2024-05-08T04:13:45.697 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34236.log 2024-05-08T04:13:45.698 INFO:teuthology.orchestra.run.smithi170.stderr: 96.4% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.36133.log.gz 2024-05-08T04:13:45.711 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37121.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37121.log.gz 2024-05-08T04:13:45.720 INFO:teuthology.orchestra.run.smithi170.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-05-08T04:13:45.720 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35410.log 2024-05-08T04:13:45.721 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34236.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34236.log.gz 2024-05-08T04:13:45.722 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.36032.log 2024-05-08T04:13:45.722 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.35410.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35410.log.gz 2024-05-08T04:13:45.723 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35089.log 2024-05-08T04:13:45.733 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.libcephsqlite.36032.log: 92.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.36084.log 2024-05-08T04:13:45.733 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.35089.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35089.log.gz 2024-05-08T04:13:45.733 INFO:teuthology.orchestra.run.smithi170.stderr: -- replaced with /var/log/ceph/ceph-client.libcephsqlite.36032.log.gz 2024-05-08T04:13:45.734 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34062.log 2024-05-08T04:13:45.734 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.libcephsqlite.36084.log: 89.4% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.36084.log.gz 2024-05-08T04:13:45.735 INFO:teuthology.orchestra.run.smithi187.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-05-08T04:13:45.738 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35453.log 2024-05-08T04:13:45.738 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34062.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34062.log.gz 2024-05-08T04:13:45.739 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34410.log 2024-05-08T04:13:45.739 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.35453.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35453.log.gz 2024-05-08T04:13:45.739 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.36215.log 2024-05-08T04:13:45.739 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34410.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34410.log.gz 2024-05-08T04:13:45.739 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35085.log 2024-05-08T04:13:45.741 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.libcephsqlite.36215.log: 91.1% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.36215.log.gz 2024-05-08T04:13:45.741 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34745.log 2024-05-08T04:13:45.741 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.35085.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35085.log.gz 2024-05-08T04:13:45.741 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35090.log 2024-05-08T04:13:45.742 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34745.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34745.log.gz 2024-05-08T04:13:45.742 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.35945.log 2024-05-08T04:13:45.743 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.35090.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35090.log.gz 2024-05-08T04:13:45.743 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36491.log 2024-05-08T04:13:45.744 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.libcephsqlite.35945.log: 90.0% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.35945.log.gz 2024-05-08T04:13:45.745 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36407.log 2024-05-08T04:13:45.745 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36491.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36491.log.gz 2024-05-08T04:13:45.746 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33962.log 2024-05-08T04:13:45.746 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36407.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36407.log.gz 2024-05-08T04:13:45.747 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.24487.log 2024-05-08T04:13:45.747 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.33962.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33962.log.gz 2024-05-08T04:13:45.748 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36449.log 2024-05-08T04:13:45.748 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph.tmp-client.admin.24487.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.24487.log.gz 2024-05-08T04:13:45.749 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37205.log 2024-05-08T04:13:45.749 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36449.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36449.log.gz 2024-05-08T04:13:45.750 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36827.log 2024-05-08T04:13:45.750 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37205.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37205.log.gz 2024-05-08T04:13:45.751 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36533.log 2024-05-08T04:13:45.751 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36827.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36827.log.gz 2024-05-08T04:13:45.752 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34749.log 2024-05-08T04:13:45.752 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36533.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36533.log.gz 2024-05-08T04:13:45.753 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37037.log 2024-05-08T04:13:45.753 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34749.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34749.log.gz 2024-05-08T04:13:45.754 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36659.log 2024-05-08T04:13:45.754 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.37037.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37037.log.gz 2024-05-08T04:13:45.755 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35088.log 2024-05-08T04:13:45.755 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36659.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36659.log.gz 2024-05-08T04:13:45.756 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36953.log 2024-05-08T04:13:45.757 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.35088.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35088.log.gz 2024-05-08T04:13:45.758 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34752.log 2024-05-08T04:13:45.759 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36953.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36953.log.gz 2024-05-08T04:13:45.759 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34020.log 2024-05-08T04:13:45.760 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34752.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34752.log.gz 2024-05-08T04:13:45.760 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36995.log 2024-05-08T04:13:45.761 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34020.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34020.log.gz 2024-05-08T04:13:45.761 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34969.log 2024-05-08T04:13:45.763 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.36995.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36995.log.gz 2024-05-08T04:13:45.763 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.libcephsqlite.36175.log 2024-05-08T04:13:45.763 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.admin.34969.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34969.log.gz 2024-05-08T04:13:45.821 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/ceph-client.libcephsqlite.36175.log: 94.7% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.36107.log.gz 2024-05-08T04:13:45.850 INFO:teuthology.orchestra.run.smithi170.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-08T04:13:46.149 INFO:teuthology.orchestra.run.smithi170.stderr: 94.6% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.36175.log.gz 2024-05-08T04:13:47.676 INFO:teuthology.orchestra.run.smithi170.stderr: 95.0% -- replaced with /var/log/ceph/ceph-client.libcephsqlite.35493.log.gz 2024-05-08T04:13:49.267 INFO:teuthology.orchestra.run.smithi170.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-08T04:13:51.065 INFO:teuthology.orchestra.run.smithi187.stderr: 92.5% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-05-08T04:13:55.715 INFO:teuthology.orchestra.run.smithi170.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-08T04:13:58.851 INFO:teuthology.orchestra.run.smithi187.stderr: 92.0% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-05-08T04:14:01.646 INFO:teuthology.orchestra.run.smithi187.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-05-08T04:14:02.249 INFO:teuthology.orchestra.run.smithi170.stderr: 91.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-05-08T04:14:03.878 INFO:teuthology.orchestra.run.smithi170.stderr: 91.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-08T04:14:03.880 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-05-08T04:14:03.881 INFO:teuthology.orchestra.run.smithi170.stderr:real 0m18.319s 2024-05-08T04:14:03.881 INFO:teuthology.orchestra.run.smithi170.stderr:user 0m50.326s 2024-05-08T04:14:03.881 INFO:teuthology.orchestra.run.smithi170.stderr:sys 0m1.608s 2024-05-08T04:14:04.625 INFO:teuthology.orchestra.run.smithi187.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-05-08T04:14:04.627 INFO:teuthology.orchestra.run.smithi187.stderr: 2024-05-08T04:14:04.627 INFO:teuthology.orchestra.run.smithi187.stderr:real 0m19.030s 2024-05-08T04:14:04.627 INFO:teuthology.orchestra.run.smithi187.stderr:user 0m52.209s 2024-05-08T04:14:04.627 INFO:teuthology.orchestra.run.smithi187.stderr:sys 0m1.651s 2024-05-08T04:14:04.627 INFO:tasks.ceph:Archiving logs... 2024-05-08T04:14:04.628 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/log/ceph to /home/teuthworker/archive/kchai-2024-05-08_00:43:58-rados-wip-kefu-testing-2024-05-06-distro-default-smithi/7696821/remote/smithi170/log 2024-05-08T04:14:04.629 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T04:14:13.024 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/log/ceph to /home/teuthworker/archive/kchai-2024-05-08_00:43:58-rados-wip-kefu-testing-2024-05-06-distro-default-smithi/7696821/remote/smithi187/log 2024-05-08T04:14:13.025 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T04:14:21.989 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-08T04:14:21.999 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-08T04:14:21.999 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T04:14:22.002 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T04:14:22.089 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-08T04:14:22.098 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-08T04:14:22.098 DEBUG:teuthology.orchestra.run.smithi170:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T04:14:22.101 DEBUG:teuthology.orchestra.run.smithi187:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T04:14:22.115 INFO:teuthology.orchestra.run.smithi170.stderr:bash: line 1: ntpq: command not found 2024-05-08T04:14:22.116 INFO:teuthology.orchestra.run.smithi187.stderr:bash: line 1: ntpq: command not found 2024-05-08T04:14:22.120 INFO:teuthology.orchestra.run.smithi170.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T04:14:22.120 INFO:teuthology.orchestra.run.smithi170.stdout:=============================================================================== 2024-05-08T04:14:22.120 INFO:teuthology.orchestra.run.smithi170.stdout:^* hv01.front.sepia.ceph.com 3 6 377 0 +457us[ +456us] +/- 68ms 2024-05-08T04:14:22.121 INFO:teuthology.orchestra.run.smithi170.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 2 -319us[ -320us] +/- 61ms 2024-05-08T04:14:22.121 INFO:teuthology.orchestra.run.smithi170.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 2 -147us[ -148us] +/- 73ms 2024-05-08T04:14:22.121 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T04:14:22.122 INFO:teuthology.orchestra.run.smithi187.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T04:14:22.122 INFO:teuthology.orchestra.run.smithi187.stdout:=============================================================================== 2024-05-08T04:14:22.122 INFO:teuthology.orchestra.run.smithi187.stdout:^* hv01.front.sepia.ceph.com 3 6 377 3 +484us[ +493us] +/- 68ms 2024-05-08T04:14:22.122 INFO:teuthology.orchestra.run.smithi187.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 2 -329us[ -329us] +/- 61ms 2024-05-08T04:14:22.123 INFO:teuthology.orchestra.run.smithi187.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 65 -147us[ -139us] +/- 72ms 2024-05-08T04:14:22.123 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T04:14:22.123 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-08T04:14:22.135 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-08T04:14:22.136 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-08T04:14:22.145 DEBUG:teuthology.orchestra.run.smithi170:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T04:14:22.229 DEBUG:teuthology.orchestra.run.smithi187:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T04:14:22.288 DEBUG:teuthology.orchestra.run.smithi170:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T04:14:22.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T04:14:22.319 DEBUG:teuthology.orchestra.run.smithi187:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T04:14:22.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T04:14:22.347 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-08T04:14:22.356 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-08T04:14:22.365 INFO:teuthology.task.internal:Duration was 832.804851 seconds 2024-05-08T04:14:22.365 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-08T04:14:22.375 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-08T04:14:22.375 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T04:14:22.378 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T04:14:22.416 INFO:teuthology.orchestra.run.smithi170.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T04:14:22.425 INFO:teuthology.orchestra.run.smithi187.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T04:14:22.801 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-08T04:14:22.802 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi170.front.sepia.ceph.com 2024-05-08T04:14:22.802 DEBUG:teuthology.orchestra.run.smithi170:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T04:14:22.864 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi187.front.sepia.ceph.com 2024-05-08T04:14:22.864 DEBUG:teuthology.orchestra.run.smithi187:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T04:14:22.899 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-08T04:14:22.899 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T04:14:22.907 DEBUG:teuthology.orchestra.run.smithi187:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T04:14:22.967 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-08T04:14:22.967 DEBUG:teuthology.orchestra.run.smithi170:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T04:14:22.975 DEBUG:teuthology.orchestra.run.smithi187:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T04:14:23.103 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-08T04:14:23.113 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-08T04:14:23.113 DEBUG:teuthology.orchestra.run.smithi170:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T04:14:23.138 DEBUG:teuthology.orchestra.run.smithi187:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T04:14:23.163 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-08T04:14:23.174 DEBUG:teuthology.orchestra.run.smithi170:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T04:14:23.181 DEBUG:teuthology.orchestra.run.smithi187:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T04:14:23.202 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = core 2024-05-08T04:14:23.226 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern = core 2024-05-08T04:14:23.287 DEBUG:teuthology.orchestra.run.smithi170:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T04:14:23.301 WARNING:teuthology.task.internal:Found coredumps on ubuntu@smithi170.front.sepia.ceph.com, flagging run as failed 2024-05-08T04:14:23.301 DEBUG:teuthology.orchestra.run.smithi187:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T04:14:23.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T04:14:23.341 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-08T04:14:23.351 INFO:teuthology.task.internal:Transferring archived files... 2024-05-08T04:14:23.352 DEBUG:teuthology.misc:Transferring archived files from smithi170:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/kchai-2024-05-08_00:43:58-rados-wip-kefu-testing-2024-05-06-distro-default-smithi/7696821/remote/smithi170 2024-05-08T04:14:23.352 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T04:14:30.661 INFO:teuthology.task.internal:Transferring binaries for coredumps... 2024-05-08T04:14:30.845 INFO:teuthology.task.internal: core looks like: /home/teuthworker/archive/kchai-2024-05-08_00:43:58-rados-wip-kefu-testing-2024-05-06-distro-default-smithi/7696821/remote/smithi170/coredump/1715141583.36175.core.gz: gzip compressed data, was "1715141583.36175.core", last modified: Wed May 8 04:14:23 2024, max compression, original size modulo 2^32 164524032 2024-05-08T04:14:30.845 INFO:teuthology.task.internal:core is compressed, try accessing gzip file ... 2024-05-08T04:14:31.586 INFO:teuthology.task.internal: core looks like: /tmp/tmp4hmpy9cf: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'sqlite3 -cmd .output /dev/null -cmd .load libcephsqlite.so -cmd pragma journal_', real uid: 1000, effective uid: 1000, real gid: 1316, effective gid: 1316, execfn: '/usr/bin/sqlite3', platform: 'x86_64' 2024-05-08T04:14:31.605 INFO:teuthology.task.internal: dump_program: sqlite3 -cmd .output /dev/null -cmd .load libcephsqlite.so -cmd pragma journal_ 2024-05-08T04:14:31.605 DEBUG:teuthology.orchestra.run.smithi170:> which 'sqlite3 -cmd .output /dev/null -cmd .load libcephsqlite.so -cmd pragma journal_' 2024-05-08T04:14:31.634 INFO:teuthology.orchestra.run.smithi170.stderr:/usr/bin/which: invalid option -- 'c' 2024-05-08T04:14:31.635 INFO:teuthology.orchestra.run.smithi170.stderr:/usr/bin/which: invalid option -- 'm' 2024-05-08T04:14:31.636 INFO:teuthology.orchestra.run.smithi170.stderr:/usr/bin/which: invalid option -- 'd' 2024-05-08T04:14:31.636 INFO:teuthology.orchestra.run.smithi170.stderr:/usr/bin/which: invalid option -- 'c' 2024-05-08T04:14:31.636 INFO:teuthology.orchestra.run.smithi170.stderr:/usr/bin/which: invalid option -- 'm' 2024-05-08T04:14:31.636 INFO:teuthology.orchestra.run.smithi170.stderr:/usr/bin/which: invalid option -- 'd' 2024-05-08T04:14:31.636 INFO:teuthology.orchestra.run.smithi170.stderr:/usr/bin/which: invalid option -- 'c' 2024-05-08T04:14:31.636 INFO:teuthology.orchestra.run.smithi170.stderr:/usr/bin/which: invalid option -- 'm' 2024-05-08T04:14:31.637 INFO:teuthology.orchestra.run.smithi170.stderr:/usr/bin/which: invalid option -- 'd' 2024-05-08T04:14:31.637 INFO:teuthology.orchestra.run.smithi170.stderr:/usr/bin/which: no .output in (/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin) 2024-05-08T04:14:31.637 INFO:teuthology.orchestra.run.smithi170.stderr:/usr/bin/which: no null in (/dev) 2024-05-08T04:14:31.637 INFO:teuthology.orchestra.run.smithi170.stderr:/usr/bin/which: no .load in (/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin) 2024-05-08T04:14:31.637 INFO:teuthology.orchestra.run.smithi170.stderr:/usr/bin/which: no libcephsqlite.so in (/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin) 2024-05-08T04:14:31.637 INFO:teuthology.orchestra.run.smithi170.stderr:/usr/bin/which: no pragma in (/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin) 2024-05-08T04:14:31.637 INFO:teuthology.orchestra.run.smithi170.stderr:/usr/bin/which: no journal_ in (/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin) 2024-05-08T04:14:31.639 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T04:14:31.639 INFO:teuthology.orchestra.run.smithi170.stdout:/usr/bin/sqlite3 2024-05-08T04:14:31.640 ERROR:teuthology.run_tasks:Manager failed: internal.archive Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/task/internal/__init__.py", line 420, in archive fetch_binaries_for_coredumps(path, rem) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/task/internal/__init__.py", line 345, in fetch_binaries_for_coredumps remote_path = remote.sh(['which', dump_program]).rstrip() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi170 with status 6: "which 'sqlite3 -cmd .output /dev/null -cmd .load libcephsqlite.so -cmd pragma journal_'" 2024-05-08T04:14:31.642 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-08T04:14:31.655 INFO:teuthology.task.internal:Not uploading archives. 2024-05-08T04:14:31.655 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-08T04:14:31.666 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-08T04:14:31.667 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T04:14:31.682 DEBUG:teuthology.orchestra.run.smithi187:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T04:14:31.698 INFO:teuthology.orchestra.run.smithi170.stdout: 262165 4 drwxr-xr-x 3 ubuntu ubuntu 4096 May 8 04:14 /home/ubuntu/cephtest 2024-05-08T04:14:31.698 INFO:teuthology.orchestra.run.smithi170.stdout: 262166 4 drwxr-xr-x 5 ubuntu ubuntu 4096 May 8 04:14 /home/ubuntu/cephtest/archive 2024-05-08T04:14:31.698 INFO:teuthology.orchestra.run.smithi170.stdout: 395306 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 04:14 /home/ubuntu/cephtest/archive/audit 2024-05-08T04:14:31.698 INFO:teuthology.orchestra.run.smithi170.stdout: 395606 116 -rw------- 1 ubuntu ubuntu 117146 May 8 04:14 /home/ubuntu/cephtest/archive/audit/audit.log.gz 2024-05-08T04:14:31.698 INFO:teuthology.orchestra.run.smithi170.stdout: 262168 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 04:14 /home/ubuntu/cephtest/archive/syslog 2024-05-08T04:14:31.698 INFO:teuthology.orchestra.run.smithi170.stdout: 265338 4 -rw-rw-rw- 1 ubuntu ubuntu 29 May 8 04:00 /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2024-05-08T04:14:31.698 INFO:teuthology.orchestra.run.smithi170.stdout: 265295 180 -rw-r--r-- 1 ubuntu ubuntu 183315 May 8 04:14 /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T04:14:31.698 INFO:teuthology.orchestra.run.smithi170.stdout: 265296 4 -rw-rw-rw- 1 ubuntu ubuntu 29 May 8 04:00 /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2024-05-08T04:14:31.699 INFO:teuthology.orchestra.run.smithi170.stdout: 262167 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 04:13 /home/ubuntu/cephtest/archive/coredump 2024-05-08T04:14:31.699 INFO:teuthology.orchestra.run.smithi170.stdout: 265422 27916 -rw------- 1 ubuntu ubuntu 164524032 May 8 04:13 /home/ubuntu/cephtest/archive/coredump/1715141583.36175.core 2024-05-08T04:14:31.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T04:14:31.700 INFO:teuthology.orchestra.run.smithi170.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-05-08T04:14:31.700 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/task/internal/__init__.py", line 420, in archive fetch_binaries_for_coredumps(path, rem) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/task/internal/__init__.py", line 345, in fetch_binaries_for_coredumps remote_path = remote.sh(['which', dump_program]).rstrip() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 97, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi170 with status 6: "which 'sqlite3 -cmd .output /dev/null -cmd .load libcephsqlite.so -cmd pragma journal_'" During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_1ae7ad82388e92a475afff437d49054826c019a1/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi170 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-05-08T04:14:31.701 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-08T04:14:31.725 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-08T04:14:31.774 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi170 with status 6: "which 'sqlite3 -cmd .output /dev/null -cmd .load libcephsqlite.so -cmd pragma journal_'" 2024-05-08T04:14:31.774 INFO:teuthology.nuke:Checking targets against current locks 2024-05-08T04:14:31.782 INFO:teuthology.orchestra.run.smithi187.stdout: 262165 4 drwxr-xr-x 3 ubuntu ubuntu 4096 May 8 04:14 /home/ubuntu/cephtest 2024-05-08T04:14:31.782 INFO:teuthology.orchestra.run.smithi187.stdout: 262166 4 drwxr-xr-x 4 ubuntu ubuntu 4096 May 8 04:14 /home/ubuntu/cephtest/archive 2024-05-08T04:14:31.782 INFO:teuthology.orchestra.run.smithi187.stdout: 405461 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 04:14 /home/ubuntu/cephtest/archive/audit 2024-05-08T04:14:31.783 INFO:teuthology.orchestra.run.smithi187.stdout: 405463 108 -rw------- 1 ubuntu ubuntu 110388 May 8 04:14 /home/ubuntu/cephtest/archive/audit/audit.log.gz 2024-05-08T04:14:31.783 INFO:teuthology.orchestra.run.smithi187.stdout: 262168 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 04:14 /home/ubuntu/cephtest/archive/syslog 2024-05-08T04:14:31.783 INFO:teuthology.orchestra.run.smithi187.stdout: 265329 4 -rw-rw-rw- 1 ubuntu ubuntu 29 May 8 04:00 /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2024-05-08T04:14:31.783 INFO:teuthology.orchestra.run.smithi187.stdout: 265295 180 -rw-r--r-- 1 ubuntu ubuntu 180851 May 8 04:14 /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T04:14:31.783 INFO:teuthology.orchestra.run.smithi187.stdout: 265296 4 -rw-rw-rw- 1 ubuntu ubuntu 29 May 8 04:00 /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2024-05-08T04:14:31.783 INFO:teuthology.orchestra.run.smithi187.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-05-08T04:14:31.807 DEBUG:teuthology.nuke:shortname: smithi170 2024-05-08T04:14:31.807 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T04:14:31.830 DEBUG:teuthology.nuke:shortname: smithi187 2024-05-08T04:14:31.830 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T04:14:31.833 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi170.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/kchai-2024-05-08_00:43:58-rados-wip-kefu-testing-2024-05-06-distro-default-smithi/7696821', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 03:51:42.950945', 'locked_by': 'scheduled_kchai@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDISdccAMpjTTpR17SreXiq5NUAYaHMBL61dw0VrbGVPmisU3BYRNBac0cDFl5tw2vHmlDZ77nLl7Dvsv2O1tW8='} 2024-05-08T04:14:31.853 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi187.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/kchai-2024-05-08_00:43:58-rados-wip-kefu-testing-2024-05-06-distro-default-smithi/7696821', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 03:51:42.952483', 'locked_by': 'scheduled_kchai@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA9FW+CA8Pfrhp6UvcPN2OnV1xZvW3bofLOIQf+CZelvy60JgzQQ8NVquLstj4GVp5uSLomwvG5Qy1KSQFa3+ig='} 2024-05-08T04:14:31.883 INFO:teuthology.orchestra.console.smithi170:Power off 2024-05-08T04:14:31.883 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-08T04:14:31.908 DEBUG:teuthology.orchestra.console.smithi170:power off output: Chassis Power Control: Down/Off 2024-05-08T04:14:31.909 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T04:14:31.925 INFO:teuthology.orchestra.console.smithi187:Power off 2024-05-08T04:14:31.926 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-05-08T04:14:31.933 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on 2024-05-08T04:14:31.948 DEBUG:teuthology.orchestra.console.smithi187:power off output: Chassis Power Control: Down/Off 2024-05-08T04:14:31.948 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T04:14:31.973 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is on 2024-05-08T04:14:35.934 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T04:14:35.974 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T04:14:36.045 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on 2024-05-08T04:14:36.085 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is on 2024-05-08T04:14:40.047 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T04:14:40.086 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T04:14:40.158 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is off 2024-05-08T04:14:40.195 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is on 2024-05-08T04:14:40.259 INFO:teuthology.orchestra.console.smithi170:Power off completed 2024-05-08T04:14:44.197 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-05-08T04:14:44.308 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is off 2024-05-08T04:14:44.409 INFO:teuthology.orchestra.console.smithi187:Power off completed 2024-05-08T04:14:44.513 INFO:teuthology.run:Summary data: description: rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_latest} tasks/libcephsqlite} duration: 832.8048510551453 failure_reason: Found coredumps on ubuntu@smithi170.front.sepia.ceph.com flavor: default owner: scheduled_kchai@teuthology status: fail success: false 2024-05-08T04:14:44.513 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T04:14:44.594 INFO:teuthology.run:FAIL